Threat intelligence assessment method, system and device and computer storage medium

Abstract

The invention discloses a threat intelligence assessment method, system and device and a computer storage medium, and relates to the technical field of network security, and the method comprises the steps: obtaining a data set composed of target threat intelligence; evaluating the data set to obtain a data set evaluation result; evaluating the target threat intelligence to obtain a data item evaluation result; and determining a target evaluation result of the target threat intelligence according to the data item evaluation result and the data set evaluation result. According to the invention, the target threat intelligence is evaluated from the overall perspective; the evaluation of each target threat intelligence is equivalent to the evaluation of the own characteristics of the target threat intelligence, so that the evaluation of the target threat intelligence from the perspective of a single body is realized; and finally, if the target assessment result of the target threat intelligence is determined according to the data item assessment result and the data set assessment result, the target threat intelligence is assessed by integrating overall and monomer angles, the angle of assessment of the target threat intelligence is expanded, and the assessment accuracy is improved.

Description

Technical Field

[0001] The present application relates to the field of network security technology, and more specifically, to a threat intelligence assessment method, system, device and computer storage medium. Background Art

[0002] With the development of information technology and the popularization of the Internet, cyberspace has become a part of modern society. However, the openness and anonymity of cyberspace also bring many security risks, and cyber threats are becoming increasingly severe. In order to detect and respond to cyber threats in a timely manner, threat intelligence can be collected and analyzed. Threat intelligence refers to information about the capabilities, intentions, and actions of cyber attackers, which can be used to predict, identify, and prevent cyber attacks.

[0003] However, due to the complexity and diversity of network threats, threat intelligence data often has problems such as uneven quality, making it difficult to conduct a comprehensive and objective quality assessment of threat intelligence data, resulting in poor reliability and credibility of threat intelligence.

[0004] In summary, how to accurately evaluate threat intelligence is an urgent problem to be solved by technical personnel in this field. Summary of the invention

[0005] The purpose of this application is to provide a threat intelligence assessment method, which can solve the technical problem of how to accurately assess threat intelligence to a certain extent. This application also provides a threat intelligence assessment system, an electronic device and a computer-readable storage medium.

[0006] In order to achieve the above objectives, this application provides the following technical solutions:

[0007] A threat intelligence assessment method, comprising:

[0008] Acquire a dataset consisting of targeted threat intelligence;

[0009] Evaluating the data set to obtain a data set evaluation result;

[0010] Evaluate the target threat intelligence to obtain a data item evaluation result;

[0011] A target evaluation result of the target threat intelligence is determined based on the data item evaluation result and the data set evaluation result.

[0012] Preferably, evaluating the data set to obtain a data set evaluation result includes:

[0013] Performing data integration consistency analysis and data relationship consistency analysis on the data set to obtain a data set consistency analysis result;

[0014] Performing data update frequency analysis and data delay analysis on the data set to obtain a data set timeliness analysis result;

[0015] Performing primary key duplication detection and unique identifier verification on the data set to obtain a data set uniqueness analysis result;

[0016] Performing business rule verification, data relationship verification, and document completeness analysis on the data set to obtain a data set normative analysis result;

[0017] Performing accuracy analysis on the data set to obtain an accuracy analysis result of the data set;

[0018] The dataset consistency analysis result, the dataset timeliness analysis result, the dataset uniqueness analysis result, the dataset standardization analysis result and the dataset accuracy analysis result are processed to generate the dataset evaluation result.

[0019] Preferably, the processing of the dataset consistency analysis result, the dataset timeliness analysis result, the dataset uniqueness analysis result, the dataset standardization analysis result and the dataset accuracy analysis result to generate the dataset evaluation result includes:

[0020] Based on the respective weights of the dataset consistency analysis result, the dataset timeliness analysis result, the dataset uniqueness analysis result, the dataset standardization analysis result and the dataset accuracy analysis result, a weight calculation is performed on the amount of consistent data, the amount of data that meets timeliness requirements, the standardization score, the amount of repeated data and the amount of incorrect data in the dataset to generate the dataset evaluation result.

[0021] Preferably, the evaluating the target threat intelligence to obtain a data item evaluation result includes:

[0022] Perform missing value ratio analysis, missing value distribution analysis and missing value impact analysis on the target threat intelligence to obtain data item integrity analysis results;

[0023] Perform data source comparison, logic verification, and sampling verification on the target threat intelligence to obtain data item accuracy analysis results;

[0024] Performing outlier detection on the target threat intelligence to obtain data item consistency analysis results;

[0025] Perform data format check, data range verification, data compliance verification, field naming clarity analysis and data dictionary verification on the target threat intelligence to obtain data item normative analysis results;

[0026] A weight operation is performed on the data item integrity analysis result, the data item accuracy analysis result, the data item consistency analysis result and the data item standardization analysis result to generate the data item evaluation result.

[0027] Preferably, performing a weight operation on the data item integrity analysis result, the data item accuracy analysis result, the data item consistency analysis result and the data item standardization analysis result to generate the data item evaluation result includes:

[0028] Based on the missing value impact coefficient of each field and the total number of fields, the number of null values ​​in each field in the data item integrity analysis result is processed to obtain an integrity score;

[0029] Based on the total amount of data sampling and the total number of fields, the amount of incorrect data in each field in the accuracy analysis result of the data item is processed to obtain an accuracy score;

[0030] Based on the total sample size and the total number of fields, the data volume of abnormal values ​​in each field in the consistency analysis result of the data item is processed to obtain a consistency score;

[0031] Based on the total number of fields, the amount of data of fields with irregular names and fields inconsistent with data fields in the data item normative analysis results is processed to obtain a normative score;

[0032] Based on the respective weights of the data item integrity analysis result, the data item accuracy analysis result, the data item consistency analysis result and the data item standardization analysis result, a weight operation is performed on the integrity score, the accuracy score, the consistency score and the standardization score to generate the data item evaluation result.

[0033] Preferably, determining a target evaluation result of the target threat intelligence according to the data item evaluation result and the data set evaluation result includes:

[0034] The sum of the data item evaluation result and the data set evaluation result is determined as the target evaluation result.

[0035] Preferably, the obtaining of a data set consisting of target threat intelligence includes:

[0036] Obtain initial threat intelligence;

[0037] Based on a pre-trained data standardization model, the initial threat intelligence is subjected to business data standardization processing and data element standardization processing to obtain the target threat intelligence;

[0038] Combining the target threat intelligence into a data set;

[0039] Among them, business terminology standards include unique term codes, Chinese names of professional terms, English names of professional terms, abbreviations, term definitions, context application examples and classification information; data element standards include business attributes, technical attributes and management attributes; the business attributes include Chinese names, English names, business definitions, business rules, value ranges, standard basis, and sensitivity; the technical attributes include data types and data formats; the management attributes include data definers, data managers, data users, business application fields and usage systems.

[0040] A threat intelligence assessment system, comprising:

[0041] A first acquisition module is used to acquire a data set consisting of target threat intelligence;

[0042] A first evaluation module is used to evaluate the data set to obtain a data set evaluation result;

[0043] A second evaluation module is used to evaluate the target threat intelligence and obtain a data item evaluation result;

[0044] The first determination module is used to determine a target evaluation result of the target threat intelligence according to the data item evaluation result and the data set evaluation result.

[0045] An electronic device, comprising:

[0046] Memory for storing computer programs;

[0047] A processor is used to implement the steps of any of the threat intelligence assessment methods described above when executing the computer program.

[0048] A computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-mentioned threat intelligence assessment methods.

[0049] The present application provides a threat intelligence assessment method, which obtains a data set composed of target threat intelligence; evaluates the data set to obtain a data set evaluation result; evaluates the target threat intelligence to obtain a data item evaluation result; and determines the target evaluation result of the target threat intelligence based on the data item evaluation result and the data set evaluation result. In the present application, since the target threat intelligence constitutes a data set, if the data set is evaluated, it is equivalent to evaluating the target threat intelligence in combination with the association between the target threat intelligence, thereby evaluating the target threat intelligence from an overall perspective; and then evaluating each target threat intelligence, it is equivalent to evaluating the characteristics of the target threat intelligence itself, thereby evaluating the target threat intelligence from a single perspective; finally, if the target evaluation result of the target threat intelligence is determined based on the data item evaluation result and the data set evaluation result, it is equivalent to evaluating the target threat intelligence from a comprehensive overall and single perspective, thereby expanding the perspective of evaluating the target threat intelligence and improving the evaluation accuracy. The present application provides a threat intelligence assessment system, electronic device, and computer-readable storage medium that also solves corresponding technical problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0051] Figure 1 A process of a threat intelligence assessment method provided in an embodiment of the present application;

[0052] Figure 2 The overall flow chart of the application plan;

[0053] Figure 3 A schematic diagram of the structure of a threat intelligence assessment system provided in an embodiment of the present application;

[0054] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0055] Figure 5 Another structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0057] See also Figure 1 , Figure 1 A flowchart of a threat intelligence assessment method provided in an embodiment of the present application.

[0058] A threat intelligence assessment method provided in an embodiment of the present application may include the following steps:

[0059] Step S101: Acquire a data set consisting of target threat intelligence.

[0060] In practical applications, you can first obtain a data set consisting of target threat intelligence, such as a data set consisting of two or more target threat intelligence. Among them, target threat intelligence refers to the threat intelligence to be evaluated, and its type can be flexibly determined according to the application scenario; and the data set refers to the collection of threat intelligence data that needs to be quality managed, which can be obtained according to business needs and after data governance, etc. Of course, the threat intelligence generated by the same device in different time periods can also be composed of a data set, etc. This application does not make specific restrictions here.

[0061] In actual applications, due to the complexity and diversity of network threats, threat intelligence data often has problems such as inconsistent formats, which makes it very difficult to use threat intelligence. In order to improve the efficiency and accuracy of threat intelligence, it is necessary to standardize threat intelligence data. Data standardization refers to converting data from different sources and formats into a unified and standardized data format for subsequent analysis and use. Through data standardization, problems such as inconsistent data formats and data redundancy can be eliminated, and data consistency and availability can be improved. In other words, in the process of obtaining a data set composed of target threat intelligence, initial threat intelligence can be obtained; based on a pre-trained data standardization model, the initial threat intelligence is subjected to business data standardization processing and data element standardization processing to obtain target threat intelligence; the target threat intelligence is composed into a data set; among them, business terminology standards include unique term codes, Chinese names of professional terms, English names of professional terms, abbreviations, term definitions, context application examples and classification information, etc. Context application examples refer to examples of the use of threat intelligence in actual application scenarios, and classification refers to the professional field or classification to which the threat intelligence belongs; data element standards include business attributes, technical attributes and management attributes; business attributes include Chinese names, English names, business definitions, business rules, value domains, standard basis, sensitivity, etc., and business rules refer to data elements. Usage rules, such as required or optional, value domain refers to the range of values ​​allowed by the data element, standard basis refers to the standards or regulations followed by the data element, and sensitivity refers to the sensitivity of the data element; technical attributes include data type and data format, data type refers to the data type of the data element, such as string, integer, etc., data format refers to the format requirements of the data element, such as date format, timestamp format, etc.; management attributes include data definer, data manager, data user, business application field and usage system, data definer refers to the organization or individual that defines the data element, data manager refers to the organization or individual responsible for maintaining the data element, data user refers to the organization or individual that uses the data element, business application field refers to the application field of the data element, and usage system refers to the systems in which the data element is used.

[0062] In specific application scenarios, the data standardization model can be composed of prompt words + RAG + big model + interface center. The prompt words are used to connect the overall process, RAG is used to do word vectorization and data standard retrieval, the big model provides vectorization capabilities as well as text understanding and generation. The big model can be a large language model, etc. The interface center is used to connect external interfaces; and in order to facilitate the understanding and processing of the big model, the standard vectorization can also be embedded in the vector knowledge base to facilitate retrieval and updating.

[0063] In specific application scenarios, in the process of obtaining target threat intelligence by standardizing business data and data elements based on the pre-trained data standardization model, the data standards connected to the data standardization model may be inconsistent. Some may only have English field names, while others may have Chinese descriptions. At this time, conversion is required. In addition to the basic field names and descriptions, the conversion also includes other information about the data, including the business attributes, technical attributes, and management attributes in the above figure. In other words, the input may only have an English or Chinese field name, but the output is the field name mapped to the standard and with other standard constraint information. On this basis, considering that the standardization mapping is performed manually, the efficiency is low and there will be inaccurate mapping conversion, because there is little information obtained and it is limited by the depth of the processing personnel's understanding of the business. There may be deviations in the understanding of business terms, and there may be inaccurate and incomplete conversion. For example: potatoes, some are called yang yams, and some are called ma yams, but they actually mean the same thing. The most extreme case is that the field name has no Chinese description, and it is not standard English, so it is impossible to understand. In order to perform standardized mapping more accurately, if the input content has Chinese fields, the meaning of this field can be analyzed based on the meaning of the Chinese field and combined with the content in the actual data, and a general word can be given. Then, the knowledge vector library can be searched to find the business terminology standard items that exceed a certain similarity threshold and have the highest similarity. If it is an English field, first understand it as the Chinese meaning, and combine it with the content in the actual data to analyze the meaning of this field, and give a general word. Then, the knowledge vector library can be searched to find the business terminology standard items that exceed a certain similarity threshold and have the highest similarity. After the first two steps get the closest business terms, they will be directly mapped to the data element standard according to the business terms, because business terms correspond to unique data element standards.

[0064] It should be noted that for new data items that cannot be mapped to existing standard data items in the data standardization model, their importance, relevance and practicality can be automatically identified and evaluated, and business terminology standards and data element standards can be updated if necessary. That is, for data items that cannot be directly mapped, preliminary identification can be performed first to try to understand their meaning and context; then the evaluation process is triggered automatically or manually, and the evaluation criteria include but are not limited to the novelty, importance and relevance of the data items to existing standards; finally, based on the evaluation results, combined with national standards, industry standards, academic papers, published books and other standard information, business terminology standards and data element standards are automatically constructed, the updated standards are incorporated into the knowledge base, and the standard vectorization table is updated synchronously, so that the newly added data items will be immediately applied to the mapping rules to ensure the consistency of subsequent data processing. In this way, the present application uses a large language model to parse and understand unstructured or semi-structured threat intelligence information, automatically identify key data elements therein, and map these elements to a unified standard data model. For new data items that cannot be mapped to existing standard data items, the system will automatically identify and evaluate their importance, relevance and practicality, and automatically update business terminology standards and data element standards when necessary. This process significantly reduces the degree of manual participation and improves the speed and accuracy of data standardization. Even when faced with threat intelligence data from different vendors and in multiple formats, this method can ensure that they are quickly integrated and standardized to form a consistent and interoperable data form.

[0065] It should also be noted that the data standardization model of this application supports multiple data source access methods, including but not limited to API interfaces, file uploads, etc. The data source may include publicly released threat intelligence reports, internal security event records, etc., and the system can be configured with a scheduled task to automatically scan the specified data source at a predetermined time interval to obtain the latest data. In addition, the input data may be non-standard. For example, the field name of "domain name" should be called domain, and the real data input is dom. In this case, the field name needs to be converted, that is, the table structure needs to be adjusted, such as through SQL statements, etc., that is, the data standardization model can construct corresponding SQL statements based on the mapping results for data conversion operations.

[0066] Step S102: Evaluate the data set to obtain a data set evaluation result.

[0067] In practical applications, after a type of threat data is standardized, the standard of the threat intelligence data set can be automatically obtained, the information required for data quality assessment can be automatically extracted from the data set standard, and targeted quality assessment indicators can be constructed and data quality assessment can be performed. Data quality assessment can include the selection of assessment indicators, the collection of quality parameters, the determination of assessment rules, the calculation of quality scores, the verification of results, and the preparation of assessment reports.

[0068] In practical applications, based on the data quality assessment process, after obtaining a data set consisting of target threat intelligence, the data set can be evaluated to analyze the target threat intelligence from a holistic perspective and obtain a data set evaluation result that can reflect the correlation or characteristics between the target threat intelligence from a holistic perspective.

[0069] In specific application scenarios, in the process of evaluating the data set and obtaining the data set evaluation results, the data set can be analyzed for data integration consistency and data relationship consistency to obtain the data set consistency analysis results; the data set can be analyzed for data update frequency and data delay to obtain the data set timeliness analysis results; the data set can be tested for primary key duplication and unique identification to obtain the data set uniqueness analysis results; the data set can be verified for business rules, data relationship verification and document completeness to obtain the data set standardization analysis results; the data set can be analyzed for accuracy to obtain the data set accuracy analysis results; the data set consistency analysis results, data set timeliness analysis results, data set uniqueness analysis results, data set standardization analysis results and data set accuracy analysis results can be processed to generate the data set evaluation results. That is, the data set can be evaluated from the perspectives of consistency, timeliness, uniqueness and standardization to obtain the corresponding data set evaluation results. Accordingly, the data set needs to be maintained regularly, including but not limited to data quality inspection, version control, etc., to ensure the accuracy and timeliness of the data set.

[0070] In specific application scenarios, in the process of processing the dataset consistency analysis results, dataset timeliness analysis results, dataset uniqueness analysis results, dataset standardization analysis results and dataset accuracy analysis results to generate dataset evaluation results, the amount of consistent data in the dataset, the amount of data that meets timeliness requirements, the standardization score, the amount of duplicate data and the amount of incorrect data can be weighted based on the respective weights of the dataset consistency analysis results, dataset timeliness analysis results, dataset uniqueness analysis results, dataset standardization analysis results and dataset accuracy analysis results to generate a dataset evaluation result.

[0071] In specific application scenarios, the data set quality indicators can be shown in Table 1. Among them, regarding consistency, there are many reasons for consistency. For example, different threat intelligence data sources may use different data formats, naming conventions, units, etc. These differences may cause data inconsistency during data integration, or data is stored in a distributed system during the integration process. Data may need to be replicated and synchronized in multiple nodes or systems. Due to network delays, system failures, etc., data may be inconsistent between different nodes; and network security data involves multiple entities, such as vulnerabilities, hash files, IOCs, etc. Each entity will have multiple different data tables to characterize the entity in multiple dimensions. There are some associations between these data tables through associated fields. It is necessary to ensure the consistency of associated fields when performing associated queries. As for uniqueness, there is only one primary key in a data set, and gradually there may be composite primary keys. When the primary key is not unique, it means that the data has data duplication, which violates the uniqueness indicator. The number of duplications will be recorded as a parameter for subsequent evaluation calculations, so it is necessary to perform duplicate value detection on all primary key fields in the data set.

[0072] Table 1 Dataset quality indicators

[0073]

[0074] In specific application scenarios, data quality assessment verifies data quality according to assessment indicators. Verification methods include: 1) System verification: Use data quality tools such as data quality management system to achieve automatic verification, such as custom SQL statement verification; 2) Manual verification: Verify data based on assessment indicators and personal professional judgment, such as data comparison, experience judgment, etc. Verification methods include 1) Full verification, that is, verify the data quality of all data involved one by one; incremental verification, that is, verify the data quality of newly added data within a specific range and time period one by one; 3) Sampling verification, that is, verify the data quality of the extracted data one by one according to the sampling plan. In the present application, a sampling verification method can be applied to perform quality assessment, that is, a data set evaluation result generation formula can be applied to perform weight calculations on the consistent data volume, the data volume that meets the timeliness requirements, the standardization score, the repeated data volume, and the incorrect data volume in the data set based on the respective weights of the data set consistency analysis results, the data set timeliness analysis results, the data set uniqueness analysis results, the data set standardization analysis results, and the data set accuracy analysis results to generate a data set evaluation result; the data set evaluation result generation formula may include:

[0075] ;

[0076] Among them, Q TRepresents the evaluation result of the dataset; WTcon represents the weight of the consistency analysis result of the dataset; ConT represents the amount of consistent data in the dataset; WTtime represents the weight of the timeliness analysis result of the dataset; Timer represents the amount of data in the dataset that meets the timeliness requirements; WTnor represents the weight of the normative analysis result of the dataset; Nort represents the normative score of the dataset; WTuniq represents the weight of the uniqueness analysis result of the dataset; UniqT represents the amount of repeated data in the dataset; WTacc represents the weight of the accuracy analysis result of the dataset; AccT represents the amount of incorrect data in the dataset; A represents the total amount of data in the dataset.

[0077] Step S103: Evaluate the target threat intelligence to obtain a data item evaluation result.

[0078] In practical applications, the target threat intelligence itself can also be evaluated to analyze the target threat intelligence from a single perspective and obtain data item evaluation results that can reflect the characteristics of the target threat intelligence from a single perspective.

[0079] In practical applications, in the process of evaluating the target threat intelligence and obtaining the data item evaluation results, the target threat intelligence can be analyzed for missing value ratio, missing value distribution and missing value impact to obtain the data item integrity analysis results; the target threat intelligence can be compared with the data source, logically verified and sampled to obtain the data item accuracy analysis results; the target threat intelligence can be detected for outliers to obtain the data item consistency analysis results; the target threat intelligence can be checked for data format, data range, data compliance, field naming clarity and data dictionary to obtain the data item standardization analysis results; the data item integrity analysis results, data item accuracy analysis results, data item consistency analysis results and data item standardization analysis results can be weighted to generate the data item evaluation results. The target threat intelligence can be evaluated from the perspectives of integrity, accuracy, consistency and standardization.

[0080] In specific application scenarios, data item quality indicators can be shown in Table 2.

[0081] Table 2 Data item quality indicators

[0082]

[0083] Accordingly, in the process of performing weight calculation on the data item integrity analysis results, data item accuracy analysis results, data item consistency analysis results and data item standardization analysis results to generate the data item evaluation results, you can:

[0084] Based on the missing value impact coefficient of each field and the total number of fields, the number of null values ​​in each field in the data item integrity analysis result is processed to obtain the integrity score; for example, the integrity score formula can be used to process the number of null values ​​in each field in the data item integrity analysis result based on the missing value impact coefficient of each field and the total number of fields to obtain the integrity score. The integrity score formula includes:

[0085] ;

[0086] in, Indicates the completeness score; kmis1, kmis2, kmis3…kmisn respectively indicate the impact coefficient of missing values ​​in the corresponding fields, which can be evaluated by the salesperson, and the value range can be 0-1, and the specific value can be 0.1, etc.; Cemp1, Cemp2, Cemp3…Cempn respectively indicate the number of null values ​​in each field; Nc indicates the total number of fields;

[0087] Based on the total amount of data sampling and the total number of fields, the incorrect amount of data in each field in the accuracy analysis result of the data item is processed to obtain the accuracy score; for example, the accuracy score formula can be used to process the incorrect amount of data in each field in the accuracy analysis result of the data item based on the total amount of data sampling and the total number of fields to obtain the accuracy score. The accuracy score formula includes:

[0088] ;

[0089] in, represents the accuracy score; Cacc1, Cacc2, Cacc3, ..., Caccn represent the amount of incorrect data in each field; Asam represents the total amount of data sampling;

[0090] Based on the total number of samples and the total number of fields, the data volume of outliers in each field in the consistency analysis results of the data items is processed to obtain a consistency score. For example, the consistency score formula can be used to process the data volume of outliers in each field in the consistency analysis results of the data items based on the total number of samples and the total number of fields to obtain a consistency score. The consistency score formula includes:

[0091] ;

[0092] in, represents the consistency score; Ccon1, Ccon2, Ccon3, ..., Cconn represent the data volume of abnormal values ​​in each field respectively;

[0093] Based on the total number of fields, the amount of data in the data item normative analysis results that are not standardized and inconsistent with the data fields is processed to obtain a normative score; for example, the normative score formula can be used to process the amount of data in the data item normative analysis results that are not standardized and inconsistent with the data fields based on the total number of fields to obtain a normative score. The normative score formula includes:

[0094] ;

[0095] Among them, Norc represents the standardization score; Cnor represents the number of fields with non-standard naming and inconsistency with the data dictionary;

[0096] Based on the respective weights of the data item integrity analysis results, data item accuracy analysis results, data item consistency analysis results and data item standardization analysis results, weight operations are performed on the integrity score, accuracy score, consistency score and standardization score to generate the data item evaluation results; based on the respective weights of the data item integrity analysis results, data item accuracy analysis results, data item consistency analysis results and data item standardization analysis results, weight operations are performed on the integrity score, accuracy score, consistency score and standardization score to generate the data item evaluation results through the data item evaluation result generation formula, and the data item evaluation result generation formula includes:

[0097] ;

[0098] Among them, Qc represents the evaluation result of the data item; WCcom represents the integrity weight of the data item; WCacc represents the accuracy weight of the data item; WCcon represents the consistency weight of the data item; and WCnor represents the standardization weight of the data item.

[0099] Step S104: Determine a target evaluation result of the target threat intelligence based on the data item evaluation result and the data set evaluation result.

[0100] In practical applications, after obtaining the data set evaluation results for evaluating the target threat intelligence from an overall perspective and the data item evaluation results for evaluating the target threat intelligence from a single perspective, the data item evaluation results and the data set evaluation results can be comprehensively analyzed to obtain the target evaluation results of the target threat intelligence, so as to comprehensively analyze the target threat intelligence from the overall perspective and the single perspective to obtain the corresponding target evaluation results.

[0101] In specific application scenarios, in the process of determining the target evaluation results of the target threat intelligence based on the data item evaluation results and the data set evaluation results, the sum of the data item evaluation results and the data set evaluation results can be determined as the target evaluation result, so as to discover problems and defects in the data through the target evaluation results, thereby facilitating the subsequent adoption of corresponding measures for improvement, such as network security upgrades for the equipment where the threat intelligence is located.

[0102] The present application provides a threat intelligence assessment method, which obtains a data set composed of target threat intelligence; evaluates the data set to obtain a data set evaluation result; evaluates the target threat intelligence to obtain a data item evaluation result; and determines the target evaluation result of the target threat intelligence based on the data item evaluation result and the data set evaluation result. In the present application, since the target threat intelligence constitutes a data set, if the data set is evaluated, it is equivalent to evaluating the target threat intelligence in combination with the association between the target threat intelligence, thereby evaluating the target threat intelligence from an overall perspective; and then evaluating each target threat intelligence, it is equivalent to evaluating the characteristics of the target threat intelligence itself, thereby evaluating the target threat intelligence from a single perspective; finally, if the target evaluation result of the target threat intelligence is determined based on the data item evaluation result and the data set evaluation result, it is equivalent to evaluating the target threat intelligence from a comprehensive overall and single perspective, thereby expanding the perspective of evaluating the target threat intelligence and improving the accuracy of the evaluation.

[0103] In order to facilitate the understanding of this application plan, it is assumed that the overall process of this application plan is as follows Figure 2 As shown in Table 3 and Table 4, the business terminology standards are shown in Table 3 and Table 4, the data element standards are shown in Table 5, the total amount of data is 10000, the weight of the data set consistency analysis result is 10, the weight of the data set timeliness analysis result is 10, the weight of the data set standardization analysis result is 5, the weight of the data set uniqueness analysis result is 15, the weight of the data set accuracy analysis result is 10, the weight of the data item integrity is 20, the weight of the data item accuracy is 10, the weight of the data item consistency is 10, and the weight of the data item standardization is 10; the total amount of data sampling is 1000. After analysis, in the data set In the data items, the amount of consistent data (total) is 9800, the amount of data that meets the timeliness requirements (total) is 9900, the standardization score is 0.95, the amount of repeated data (total) is 100, and the amount of incorrect data (total / sampled) is 500 (total) / 50 (sampled); in the data items, the total number of fields is 30, the impact coefficient of missing values ​​in each field is 0.1, the number of null values ​​in each field (total) is 200, the amount of incorrect data in each field (sampled) is 20, the amount of data with abnormal values ​​in each field (total / sampled) is 20, and the number of fields with irregular naming and inconsistent with the data dictionary is 2;

[0104] Table 3 Business terminology standards 1

[0105]

[0106] Table 4 Business terminology standards 2

[0107]

[0108] Table 5 Data element standards

[0109]

[0110] The target evaluation result is:

[0111] ;

[0112] After substituting the values:

[0113] ;

[0114] The final calculation is:

[0115] .

[0116] See also Figure 3 , Figure 3 A schematic diagram of the structure of a threat intelligence assessment system provided in an embodiment of the present application.

[0117] A threat intelligence assessment system provided in an embodiment of the present application may include:

[0118] A first acquisition module 101 is used to acquire a data set consisting of target threat intelligence;

[0119] A first evaluation module 102 is used to evaluate the data set and obtain a data set evaluation result;

[0120] The second evaluation module 103 is used to evaluate the target threat intelligence and obtain a data item evaluation result;

[0121] The first determination module 104 is used to determine a target evaluation result of the target threat intelligence according to the data item evaluation result and the data set evaluation result.

[0122] In a threat intelligence assessment system provided in an embodiment of the present application, a first assessment module may include:

[0123] The first analysis unit is used to perform data integration consistency analysis and data relationship consistency analysis on the data set to obtain the data set consistency analysis result; perform data update frequency analysis and data delay analysis on the data set to obtain the data set timeliness analysis result; perform primary key duplication detection and unique identifier verification on the data set to obtain the data set uniqueness analysis result; perform business rule verification, data relationship verification and document completeness analysis on the data set to obtain the data set standardization analysis result; perform accuracy analysis on the data set to obtain the data set accuracy analysis result;

[0124] The first processing unit is used to process the data set consistency analysis results, the data set timeliness analysis results, the data set uniqueness analysis results, the data set standardization analysis results and the data set accuracy analysis results to generate a data set evaluation result.

[0125] In a threat intelligence assessment system provided by an embodiment of the present application, the first processing unit is specifically used to: based on the respective weights of the data set consistency analysis results, the data set timeliness analysis results, the data set uniqueness analysis results, the data set standardization analysis results and the data set accuracy analysis results, perform weighted operations on the amount of consistent data in the data set, the amount of data that meets the timeliness requirements, the standardization score, the amount of repeated data and the amount of incorrect data to generate a data set assessment result.

[0126] In a threat intelligence assessment system provided in an embodiment of the present application, the second assessment module may include:

[0127] The second analysis unit is used to perform missing value ratio analysis, missing value distribution analysis and missing value impact analysis on the target threat intelligence to obtain data item integrity analysis results; perform data source comparison, logic verification and sampling verification on the target threat intelligence to obtain data item accuracy analysis results; perform outlier detection on the target threat intelligence to obtain data item consistency analysis results;

[0128] The second processing unit is used to perform data format check, data range verification, data compliance verification, field naming clarity analysis and data dictionary verification on the target threat intelligence to obtain data item normative analysis results; perform weight calculations on the data item integrity analysis results, data item accuracy analysis results, data item consistency analysis results and data item normative analysis results to generate data item evaluation results.

[0129] A threat intelligence assessment system provided by an embodiment of the present application, wherein the second processing unit is specifically used to: process the number of null values ​​in each field in the data item integrity analysis result based on the missing value impact coefficient of each field and the total number of fields to obtain an integrity score; process the incorrect data amount in each field in the data item accuracy analysis result based on the total data sampling amount and the total number of fields to obtain an accuracy score; process the data amount of abnormal values ​​in each field in the data item consistency analysis result based on the total sampling amount and the total number of fields to obtain a consistency score; process the data amount of fields with irregular names and inconsistent fields with data fields in the data item standardization analysis result based on the total number of fields to obtain a standardization score; and perform weight calculations on the integrity score, accuracy score, consistency score and standardization score based on the respective weights of the data item integrity analysis result, the data item accuracy analysis result, the data item consistency analysis result and the data item standardization analysis result to generate a data item assessment result.

[0130] In a threat intelligence assessment system provided by an embodiment of the present application, a first determination module may include:

[0131] The first determining unit is used to determine the sum of the data item evaluation result and the data set evaluation result as the target evaluation result.

[0132] In a threat intelligence assessment system provided in an embodiment of the present application, a first acquisition module may include:

[0133] A first acquisition unit, used to acquire initial threat intelligence;

[0134] A third processing unit is used to perform business data standardization processing and data element standardization processing on the initial threat intelligence based on a pre-trained data standardization model to obtain target threat intelligence;

[0135] A fourth processing unit, configured to form a data set from target threat intelligence;

[0136] Among them, business terminology standards include unique term codes, Chinese names of professional terms, English names of professional terms, abbreviations, term definitions, contextual application examples and classification information; data element standards include business attributes, technical attributes and management attributes; business attributes include Chinese names, English names, business definitions, business rules, value ranges, standard basis, and sensitivity; technical attributes include data types and data formats; management attributes include data definers, data managers, data users, business application fields and usage systems.

[0137] The present application also provides an electronic device and a computer-readable storage medium, both of which have the corresponding effects of the threat intelligence assessment method provided in the embodiment of the present application. Figure 4 , Figure 4A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0138] An electronic device provided in an embodiment of the present application includes a memory 201 and a processor 202. The memory 201 stores a computer program. When the processor 202 executes the computer program, the steps of the threat intelligence assessment method described in any of the above embodiments are implemented.

[0139] See also Figure 5 , another electronic device provided in the embodiment of the present application may also include: an input port 203 connected to the processor 202, for transmitting commands input from the outside to the processor 202; a display unit 204 connected to the processor 202, for displaying the processing results of the processor 202 to the outside; a communication module 205 connected to the processor 202, for realizing communication between the electronic device and the outside. The display unit 204 can be a display panel, a laser scanning display, etc.; the communication method adopted by the communication module 205 includes but is not limited to mobile high-definition link technology (Mobile High-Definition Link, MHL), Universal Serial Bus (Universal Serial Bus, USB), High-Definition Multimedia Interface (High-DefinitionMultimedia Interface, HDMI), wireless connection: wireless fidelity technology (WIreless Fidelity, WiFi), Bluetooth communication technology, low-power Bluetooth communication technology, and communication technology based on IEEE802.11s.

[0140] An embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the threat intelligence assessment method described in any of the above embodiments are implemented.

[0141] The computer-readable storage medium involved in this application includes random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs (Compact Disc Read-Only Memory), or any other form of storage medium known in the technical field.

[0142] For the description of the relevant parts of the threat intelligence assessment system, electronic device, and computer-readable storage medium provided in the embodiments of the present application, please refer to the detailed description of the corresponding parts in the threat intelligence assessment method provided in the embodiments of the present application, which will not be repeated here. In addition, the parts of the above-mentioned technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive elaboration.

[0143] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0144] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A threat intelligence assessment method, characterized in that: include: Acquire a dataset consisting of targeted threat intelligence; Evaluating the data set to obtain a data set evaluation result; Evaluate the target threat intelligence to obtain a data item evaluation result; A target evaluation result of the target threat intelligence is determined based on the data item evaluation result and the data set evaluation result.

2. The method according to claim 1, characterized in that The step of evaluating the data set to obtain a data set evaluation result includes: Performing data integration consistency analysis and data relationship consistency analysis on the data set to obtain a data set consistency analysis result; Performing data update frequency analysis and data delay analysis on the data set to obtain a data set timeliness analysis result; Performing primary key duplication detection and unique identifier verification on the data set to obtain a data set uniqueness analysis result; Performing business rule verification, data relationship verification, and document completeness analysis on the data set to obtain a data set normative analysis result; Performing accuracy analysis on the data set to obtain an accuracy analysis result of the data set; The dataset consistency analysis result, the dataset timeliness analysis result, the dataset uniqueness analysis result, the dataset standardization analysis result and the dataset accuracy analysis result are processed to generate the dataset evaluation result.

3. The method according to claim 2, characterized in that The processing of the dataset consistency analysis result, the dataset timeliness analysis result, the dataset uniqueness analysis result, the dataset standardization analysis result, and the dataset accuracy analysis result to generate the dataset evaluation result includes: Based on the respective weights of the dataset consistency analysis result, the dataset timeliness analysis result, the dataset uniqueness analysis result, the dataset standardization analysis result and the dataset accuracy analysis result, a weight calculation is performed on the amount of consistent data, the amount of data that meets timeliness requirements, the standardization score, the amount of repeated data and the amount of incorrect data in the dataset to generate the dataset evaluation result.

4. The method according to claim 3, characterized in that The evaluating the target threat intelligence to obtain a data item evaluation result includes: Perform missing value ratio analysis, missing value distribution analysis and missing value impact analysis on the target threat intelligence to obtain data item integrity analysis results; Perform data source comparison, logic verification, and sampling verification on the target threat intelligence to obtain data item accuracy analysis results; Performing outlier detection on the target threat intelligence to obtain data item consistency analysis results; Perform data format check, data range verification, data compliance verification, field naming clarity analysis and data dictionary verification on the target threat intelligence to obtain data item normative analysis results; A weight operation is performed on the data item integrity analysis result, the data item accuracy analysis result, the data item consistency analysis result and the data item standardization analysis result to generate the data item evaluation result.

5. The method according to claim 4, characterized in that The step of performing a weight operation on the data item integrity analysis result, the data item accuracy analysis result, the data item consistency analysis result, and the data item standardization analysis result to generate the data item evaluation result includes: Based on the missing value impact coefficient of each field and the total number of fields, the number of null values ​​in each field in the data item integrity analysis result is processed to obtain an integrity score; Based on the total amount of data sampling and the total number of fields, the amount of incorrect data in each field in the accuracy analysis result of the data item is processed to obtain an accuracy score; Based on the total sample size and the total number of fields, the data volume of abnormal values ​​in each field in the consistency analysis result of the data item is processed to obtain a consistency score; Based on the total number of fields, the amount of data of fields with irregular names and fields inconsistent with data fields in the data item normative analysis results is processed to obtain a normative score; Based on the respective weights of the data item integrity analysis result, the data item accuracy analysis result, the data item consistency analysis result and the data item standardization analysis result, a weight operation is performed on the integrity score, the accuracy score, the consistency score and the standardization score to generate the data item evaluation result.

6. The method according to claim 5, characterized in that Determining a target evaluation result of the target threat intelligence according to the data item evaluation result and the data set evaluation result includes: The sum of the data item evaluation result and the data set evaluation result is determined as the target evaluation result.

7. The method according to any one of claims 1 to 6, characterized in that: The obtaining of a data set consisting of target threat intelligence includes: Obtain initial threat intelligence; Based on a pre-trained data standardization model, the initial threat intelligence is subjected to business data standardization processing and data element standardization processing to obtain the target threat intelligence; Combining the target threat intelligence into a data set; Among them, business terminology standards include unique term codes, Chinese names of professional terms, English names of professional terms, abbreviations, term definitions, context application examples and classification information; data element standards include business attributes, technical attributes and management attributes; the business attributes include Chinese names, English names, business definitions, business rules, value ranges, standard basis, and sensitivity; the technical attributes include data types and data formats; the management attributes include data definers, data managers, data users, business application fields and usage systems.

8. A threat intelligence assessment system, characterized in that: include: A first acquisition module is used to acquire a data set consisting of target threat intelligence; A first evaluation module is used to evaluate the data set to obtain a data set evaluation result; A second evaluation module is used to evaluate the target threat intelligence and obtain a data item evaluation result; The first determination module is used to determine a target evaluation result of the target threat intelligence according to the data item evaluation result and the data set evaluation result.

9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the threat intelligence assessment method as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the threat intelligence assessment method according to any one of claims 1 to 7 are implemented.

Images