Quantitative evaluation method for network threat intelligence quality

Abstract

The invention provides a quantitative evaluation method for network threat intelligence quality, and relates to the technical field of network security, and the method comprises the steps: determining the quality evaluation field of network threat intelligence, the quality evaluation field having a plurality of evaluation samples; obtaining M quality factors corresponding to the quality evaluation field, randomly selecting N quality factors as target quality factors, and based on the target quality factors, carrying out weight sorting on the evaluation sample to obtain a sorting sequence, and then establishing a network threat intelligence quality evaluation model; and applying the network threat intelligence quality evaluation model to a preset field, and performing quantitative evaluation on the network threat intelligence quality evaluation model to obtain a quality evaluation score of the preset field. By clearly defining and quantifying the assessment indexes of the threat intelligence, the influence of subjective judgment of experts on the network threat intelligence quality assessment result is reduced, and the accuracy and comprehensiveness of the network threat intelligence quality assessment result are improved.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and in particular to a method for quantitatively evaluating the quality of network threat intelligence. Background Art

[0002] As cybersecurity threats become increasingly complex and diverse, cyber threat intelligence has become an indispensable part of an organization's defense strategy. Cyber ​​threat intelligence can come from a variety of channels, such as public sources, secret sources, social media, and honeypot traps. This has led to an increasing diversity and massiveness of cyber threat intelligence sources, posing challenges to the effective assessment of the quality of cyber threat intelligence. Therefore, how to quantitatively evaluate the quality of cyber threat intelligence is of great significance.

[0003] At present, network threat intelligence assessment methods generally use crawlers, API interfaces and other means to automatically collect relevant network threat intelligence data, fuse network threat intelligence data from different sources, use predefined rules to perform preliminary screening and evaluation on the fused data, or directly rely on experts' subjective experience and predefined rules to judge the reliability and relevance of relevant network threat intelligence data, so as to improve the processing speed of network threat intelligence, which has certain application value in practice.

[0004] However, existing methods often have strong subjectivity in scoring, cannot fully cover multiple important dimensions of threat intelligence quality, and have large human errors, which to some extent affect the accuracy and comprehensiveness of network threat intelligence quality assessment results. Therefore, it is urgent to provide a solution to improve the above problems. Summary of the invention

[0005] The purpose of the present invention is to provide a method for quantitatively evaluating the quality of network threat intelligence, so as to improve the problem that when evaluating the quality of network threat intelligence in existing methods, the high human subjectivity leads to low comprehensiveness and accuracy in the evaluation of the quality of network threat intelligence.

[0006] The present invention provides a method for quantitatively evaluating the quality of network threat intelligence using the following technical solutions:

[0007] Acquire a quality assessment field of network threat intelligence, wherein the quality assessment field has a plurality of assessment samples, and the quality assessment field includes: raw data, intelligence, and service providers;

[0008] Obtain M quality factors corresponding to the quality assessment field, arbitrarily select N of the quality factors as target quality factors, and weight the assessment samples based on the target quality factors to obtain a sorting sequence, and establish a network threat intelligence quality assessment model based on the sorting sequence; wherein, N <M;

[0009] The network threat intelligence quality assessment model is applied to a preset field, and the network threat intelligence quality assessment model is quantitatively evaluated based on objectivity, subjectivity, performance, behavior and accuracy to obtain a quality assessment score for the preset field.

[0010] The beneficial effects of the quantitative evaluation method of network threat intelligence quality provided by the present invention are that, firstly, the subjective judgment of experts is converted into quantifiable values, which reduces the influence of subjective judgment in the process of network threat intelligence quality evaluation, thereby improving the objectivity of the evaluation results; secondly, the evaluation indicators of threat intelligence are clearly defined and quantified, which improves the problem of lack of clear standards in traditional evaluation methods. The present invention ensures the comprehensiveness and accuracy of network threat intelligence quality evaluation by establishing a comprehensive set of quantitative evaluation standards and applying the evaluation standards to the preset industrial fields.

[0011] Optionally, the process of obtaining M quality factors corresponding to the quality assessment field includes: when the quality assessment field is raw data, the quality factors corresponding to the raw data include at least one of accuracy, timeliness, completeness, consistency, relevance, feasibility and value; when the quality assessment field is intelligence data, the quality factors corresponding to the intelligence data include at least one of accuracy, clarity, availability, relevance, timeliness, completeness, digestibility, and credibility; when the quality assessment field is a service provider, the quality factors corresponding to the service provider include at least one of user evaluation, overall quality of shared information, traceability, provability, and service price.

[0012] Optionally, the process of weighting and sorting the evaluation samples to obtain a sorting sequence includes:

[0013] The process of weighting and sorting the evaluation samples to obtain a sorting sequence includes:

[0014] After obtaining the relative importance of each target quality factor, a score calculation is performed to obtain a pair comparison matrix, and all elements in the pair comparison matrix are subjected to standard normalization processing to obtain a standardized pair comparison matrix;

[0015] Calculating a single ranking weight of the standardized pairwise comparison matrix, and calculating a maximum eigenvalue of the standardized pairwise comparison matrix according to the single ranking weight;

[0016] Calculating a consistency ratio according to the maximum eigenvalue, and dynamically adjusting all elements in the standardized pairwise comparison matrix according to the consistency ratio to obtain an updated standardized pairwise comparison matrix;

[0017] The single ranking weight of each row element in the updated standardized pairwise comparison matrix is ​​calculated, and the total ranking weight is obtained after weighted summation. The total ranking weight is sorted from large to small to obtain the ranking sequence of the evaluation samples corresponding to each quality assessment field.

[0018] Optionally, each element in the pairwise comparison matrix represents the relative importance of the row target quality factor relative to the column target quality factor, and the greater the relative importance, the more important the row target quality factor is relative to the column target quality factor.

[0019] Optionally, in the process of dynamically adjusting all elements in the standardized pairwise comparison matrix according to the consistency ratio, a convergence threshold needs to be set for consistency judgment. When the consistency ratio is greater than the convergence threshold, the element values ​​in the pairwise comparison matrix need to be changed again until the consistency ratio is less than the convergence threshold.

[0020] Optionally, the preset domain is an industrial domain, which includes an industrial domain set, information technology assets and unstructured products, wherein the organization operates on the industrial domain set.

[0021] Optionally, the process of quantitatively evaluating the cyber threat intelligence quality assessment model based on objectivity, subjectivity, performance, behavior, and accuracy includes:

[0022] A first document similarity function is obtained based on the cosine value of the information technology asset and the unstructured network threat intelligence product, and a second document similarity function is obtained based on the Jaccard similarity of the industrial domain set and the unstructured network threat intelligence product;

[0023] Obtaining an industrial domain function based on the relative independence of the first document similarity function and the second document similarity function, and obtaining an accuracy score based on the algorithmic determinism of the industrial domain function;

[0024] Obtaining a performance score based on the time complexity of the first document similarity function and the second document similarity function;

[0025] Based on the subjectivity and objectivity of the data of the industrial domain function and the subjectivity and objectivity of the measurement method, the subjective and objective scores are obtained respectively, and the behavioral score is obtained after sensitivity analysis of the industrial domain function.

[0026] Optionally, in the process of obtaining the performance score, a comprehensive time complexity is obtained based on the time complexity of the first document similarity function and the second document similarity function, and the performance score is obtained based on the comprehensive time complexity.

[0027] Optionally, the comprehensive time complexity is the product of the time complexity of the first document similarity function and the second document similarity function. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A flowchart of a method for quantitatively evaluating the quality of network threat intelligence provided by the present invention is shown;

[0029] Figure 2 It represents the threat intelligence quality assessment indicator system diagram provided by the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be the usual meanings understood by people with general skills in the field to which the present invention belongs. The words "including" and similar words used in the present invention mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects.

[0031] In some embodiments, see Figure 1 , a flow chart of a method for quantitatively evaluating the quality of network threat intelligence provided by the present invention comprises the following steps:

[0032] S1. Acquire a quality assessment domain of network threat intelligence, wherein the quality assessment domain has multiple assessment samples, and the quality assessment domain includes: raw data, intelligence, and service providers;

[0033] S2. Obtain M quality factors corresponding to the quality assessment field, arbitrarily select N of the quality factors as target quality factors, and weight-sort the assessment samples based on the target quality factors to obtain a sorting sequence, and establish a network threat intelligence quality assessment model based on the sorting sequence; wherein, N <M;

[0034] S3. Apply the network threat intelligence quality assessment model to a preset field, and obtain a quality assessment score for the preset field after quantitatively evaluating the network threat intelligence quality assessment model based on objectivity, subjectivity, performance, behavior, and accuracy.

[0035] In some embodiments, see Figure 2, represents the network threat intelligence quality evaluation index system diagram provided by the present invention, specifically including: first, determining the evaluation field of the network threat intelligence quality to be evaluated, the evaluation field consists of raw data, intelligence, and service providers. Secondly, different evaluation fields have different quality factors. The raw data field has quality factors such as value, accuracy, and completeness, the intelligence field has quality factors such as relevance and timeliness, and the service provider has quality factors such as user evaluation, traceability, and service price. Management decision makers can select any one of the evaluation fields as the quality evaluation field according to actual needs.

[0036] In some embodiments, when executing step S1 to obtain the quality assessment field of network threat intelligence, the quality assessment field is any one of the original data, intelligence and service provider, and each quality assessment field has multiple corresponding assessment samples, specifically including: the original data has multiple original data samples, the intelligence has multiple intelligence samples, and the service provider has multiple service provider samples.

[0037] In some embodiments, when executing step S2 to obtain M quality factors corresponding to the quality assessment field, when the quality assessment field is raw data, the quality factors corresponding to the raw data include at least one of accuracy, timeliness, completeness, consistency, relevance, feasibility and value.

[0038] Further, when the quality assessment field is intelligence data, the quality factor corresponding to the intelligence data includes at least one of accuracy, clarity, availability, relevance, timeliness, completeness, digestibility, and credibility;

[0039] Furthermore, when the quality assessment domain is a service provider, the quality factors corresponding to the service provider include at least one of user evaluation, overall quality of shared information, traceability, provability, and service price.

[0040] In some embodiments, the process of performing weight sorting on the evaluation samples to obtain a sorting sequence in step S2 includes:

[0041] S2-1, obtaining the relative importance of each target quality factor, performing score calculation, and obtaining a pair comparison matrix, and performing standard normalization processing on all elements in the pair comparison matrix to obtain a standardized pair comparison matrix;

[0042] S2-2, calculating the single sorting weight of the standardized pairwise comparison matrix, and calculating the maximum eigenvalue of the standardized pairwise comparison matrix according to the single sorting weight;

[0043] S2-3, calculating the consistency ratio according to the maximum eigenvalue, and dynamically adjusting all elements in the standardized pairwise comparison matrix according to the consistency ratio to obtain an updated standardized pairwise comparison matrix;

[0044] S2-4. Calculate the single ranking weight of each row element in the updated standardized pairwise comparison matrix, and obtain the total ranking weight after weighted summation. Then, sort the total ranking weight from large to small to obtain the ranking sequence of the evaluation samples corresponding to each quality assessment field.

[0045] Specifically, when executing step S2-1 to construct the pairwise comparison matrix, each element in the pairwise comparison matrix represents the relative importance of the row target quality factor relative to the column target quality factor, and the greater the relative importance, the more important the row target quality factor is relative to the column target quality factor. The relative importance is taken in the range of 1 to 9.

[0046] In fact, a relative importance of 1 means that the row target quality factor and the column target quality factor are equally important, and 9 means that the row target quality factor is extremely important compared to the column target quality factor. Since it is a paired comparison, the importance of row target quality factor i relative to column target quality factor j is X ij , then the importance of column target quality factor j relative to row target quality factor i is

[0047] Furthermore, in executing step S2-1, the process of performing standard normalization processing on all elements in the pairwise comparison matrix to obtain a standardized pairwise comparison matrix includes:

[0048] Get the sum of the elements in each column of the pairwise comparison matrix, and divide each element by the sum of the column to obtain a standardized pairwise comparison matrix with a sum of 1 in each column. The values ​​of the elements in the standardized pairwise comparison matrix are:

[0049]

[0050] Among them, r ij represents the value of the elements in the standardized pairwise comparison matrix, represents the sum of the elements in each column of the pairwise comparison matrix, n is the order of the pairwise comparison matrix, x ij Represents the element values ​​in the pairwise comparison matrix.

[0051] Further, when executing step S2-2 to calculate the single sorting weight of the standardized pairwise comparison matrix, the single sorting weight is the average value of each row element in the standardized pairwise comparison matrix, expressed as:

[0052]

[0053] Among them, ωi is a single sorting weight.

[0054] Further, when executing step S2-2, according to the single sorting weight ω i When calculating the maximum eigenvalue of the standardized pairwise comparison matrix, the maximum eigenvalue λ max It is expressed as:

[0055]

[0056] Among them, (Xω) i =λ i ω i ,λ i Represents the i-th eigenvalue of the standardized pairwise comparison matrix.

[0057] Furthermore, the process of dynamically adjusting all elements in the standardized pairwise comparison matrix according to the consistency ratio in step S2-3 includes:

[0058] Obtain the consistency measurement index CI of the pairwise comparison matrix, the mathematical expression is:

[0059]

[0060] Among them, λ max (X) is the largest eigenvalue of the pairwise comparison matrix X.

[0061] The random consistency index RI is obtained according to the order of the pairwise comparison matrix. In fact, RI is a preset indicator that depends on the order of the matrix. It is the average CI value calculated from a large number of randomly filled pairwise comparison matrices. Matrices of different orders have different RI values.

[0062] The consistency ratio CR is obtained based on the consistency degree measurement index CI and the random consistency index RI, where the consistency ratio CR is expressed as:

[0063]

[0064] In some embodiments, when executing step S2-3, in the process of dynamically adjusting all elements in the standardized paired comparison matrix according to the consistency ratio, a convergence threshold needs to be set for consistency judgment. When the consistency ratio is greater than the convergence threshold, it is necessary to change the element value in the paired comparison matrix again until the consistency ratio is less than the convergence threshold.

[0065] Specifically, the convergence threshold is 0.1. When the consistency ratio CR is less than 0.1, it means that the consistency of the pairwise comparison matrix can be accepted. When the consistency ratio CR is greater than or equal to 0.1, it is necessary to readjust the elements in the pairwise comparison matrix until the consistency ratio CR is less than 0.1.

[0066] Furthermore, when the total ranking weight is obtained after weighted summation during the execution of step S2-4, the expression of the total ranking weight is:

[0067]

[0068] Among them, b i represents the total ranking weight of the i-th quality assessment field, a ij represents the single ranking weight of the i-th quality assessment field for the j-th target quality factor, and m represents the number of target quality factors, which is the same as the order n of the pairwise comparison matrix.

[0069] In fact, after sorting the total sorting weight from large to small, a sorting sequence of evaluation samples corresponding to each quality assessment field is obtained. According to the sorting sequence, the security administrator can select the best evaluation sample in each quality assessment field, that is, select the first one in the corresponding sorting sequence in each quality assessment field.

[0070] In some embodiments, when executing step S3, the preset domain is an industrial domain, and the industrial domain includes an industrial domain set, information technology assets, and network unstructured products, wherein the organization C is in a limited industrial domain set D = {d1, d2, ... d m}.

[0071] Specifically, network unstructured products P u For emails, web forums and social media, blogs and news articles, technical documents, pdf documents, etc., it is represented as: u = {P1,P2,...,P q}, information technology assets I is a finite sequence, expressed as: I = {i1,i2,...,i g}, where i1,i2,...,i g Represents each information technology asset, g represents the number of finite sequences, and q represents the number of network unstructured products.

[0072] In some embodiments, the process of performing a quantitative evaluation of the network threat intelligence quality evaluation model based on objectivity, subjectivity, performance, behavior, and accuracy in step S3 includes:

[0073] S3-1, obtaining a first document similarity function based on the cosine value of information technology assets and unstructured network threat intelligence products, and obtaining a second document similarity function based on the Jaccard similarity of the industrial domain set and the unstructured network threat intelligence products;

[0074] S3-2, obtaining an industrial domain function based on the relative independence of the first document similarity function and the second document similarity function, and obtaining an accuracy score based on the algorithmic determinism of the industrial domain function;

[0075] S3-3, obtaining a performance score based on the time complexity of the first document similarity function and the second document similarity function;

[0076] S3-4. Obtain subjective and objective scores based on the subjectivity and objectivity of the data of the industrial domain function and the subjectivity and objectivity of the measurement method, and obtain the behavioral score after performing a sensitivity analysis on the industrial domain function.

[0077] Specifically, the process of obtaining the first document similarity function based on the cosine value of the information technology asset and the unstructured network threat intelligence product in step S3-1 is as follows:

[0078]

[0079] Among them, d I and I and P respectively un Document vector conversion.

[0080] Furthermore, the process of obtaining the second document similarity function based on the Jaccard similarity of the industrial domain set and the unstructured network threat intelligence product is as follows:

[0081]

[0082] Among them, d D Represents the document vector transformation of the industrial domain set D,

[0083] Furthermore, when executing step S3-2, the process of obtaining the industrial domain function based on the relative independence of the first document similarity function and the second document similarity function is:

[0084] Based on the first document similarity function F1(I,P un ) and the second document similarity function F2(D,P un ) is used to calculate the relative independence of the industrial domain function F(X), which can be expressed as:

[0085]

[0086] The industrial domain function F(X) is obtained by converting the first document similarity function F1(I,P un ) and the second document similarity function F2(D,P un ) is obtained by multiplying, and F1(I,P un ) is calculated by the cosine function, F2(D,Pun ) is calculated by the Jaccard similarity function. Since the cosine function and the Jaccard similarity function are deterministic, the data is objective, and the variable D = {d1, d2, ..., d m}Determined by human factors, which determine the industrial field of organization C, so the measurement method is subjective, that is, it is manifested as: subjective measurement of objective data.

[0087] In fact, accuracy is used to describe the predicted value M c With the actual value M r The difference between the predicted value and the true value is called the deviation, which is represented by b. When evaluating the quality of network threat intelligence, the accuracy score A=A(b), A∈[0,1], is defined. The accuracy score can convert the deviation b into a value between 0 and 1, where 0 represents complete inaccuracy and the maximum deviation, and 1 represents complete accuracy without any deviation. The mathematical relationship between the predicted value and the true value is: r =A(b)M c .

[0088] Ideally, A=A(b)=1, indicating no deviation. In the process of calculating the industrial domain function F(X), F(X) is obtained through a specific mathematical calculation formula rather than through an experiment with uncertainty. The acquisition process is deterministic, so no deviation is introduced when calculating F(X), and the accuracy score A=1.

[0089] Furthermore, in the process of obtaining subjective and objective scores based on the subjectivity and objectivity of the data of the industrial domain function and the subjectivity and objectivity of the measurement method when executing step S3-4, the network threat intelligence quality assessment is divided into four subjective / objective levels, as shown in Table 1:

[0090] Table 1 Subjectivity / objectivity level of cyber threat intelligence quality assessment

[0091]

[0092] Furthermore, OO was assigned 4 points, SO was assigned 3 points, OS was assigned 2 points, and SS was assigned 1 point.

[0093] Since the way to obtain the industrial domain function measurement F(X) is a subjective measurement of objective data, according to Table 2, the subjective and objective score is 3 points.

[0094] In some embodiments, in the process of executing step S3-3 to obtain the performance score, the comprehensive time complexity is obtained based on the time complexity of the first document similarity function and the second document similarity function, and the performance score is obtained based on the comprehensive time complexity. The time complexity and the corresponding score are shown in Table 2.

[0095] Table 2 Time complexity corresponding to the scoring table

[0096]

[0097]

[0098] In fact, the first document similarity function F1(I,P un ) has a time complexity of O(n), because in calculating the second document similarity function F2(D,P un ), avoiding logical and intersection operations, F2(D,P un ) is O(1), so the comprehensive time complexity is the product of the time complexity of the first document similarity function and the second document similarity function, expressed as: O(n) × O(1)=O(n), according to Table 2, the performance score P=3.

[0099] Furthermore, the baseline behavior factor B baseline It is expressed as:

[0100] B baseline =ω O ·O+ω S ·S+ω P ·P+ω A A;

[0101] Among them, O represents subjectivity, S represents objectivity, P represents performance score, A represents accuracy score, ω O represents the subjective weight, ω S represents the objectivity weight, ω P represents the performance score weight, ω A Represents the accuracy score weight.

[0102] Furthermore, the baseline values ​​of O, S, P, and A are increased by 10% respectively to obtain the updated behavior factor B new,i , expressed as:

[0103] B new,i =ω O ·O'+ω S ·S'+ω P ·P'+ω A ·A';

[0104] Among them, O' represents the updated subjectivity, S' represents the updated objectivity, P' represents the updated performance score, and A' represents the updated accuracy score.

[0105] Furthermore, based on the baseline behavior factor B baseline and the updated behavior factor B new,i Get Sensitivity Index i , expressed as:

[0106]

[0107] Furthermore, the score of the behavioral factor is the sensitivity index. i The value of .

[0108] Furthermore, the accuracy score, performance score, subjective and objective score, and behavior score were obtained comprehensively, namely: the accuracy score A is 1 point, the subjective and objective score OS is 3 points, the performance score P is 3 points, and the behavior score is Sensitivity Index i The value of completes the quantitative evaluation process of the quality of network threat intelligence.

[0109] Although the embodiments of the present invention are described in detail above, it is obvious to those skilled in the art that various modifications and variations can be made to these embodiments. However, it should be understood that such modifications and variations are within the scope and spirit of the present invention as described in the claims. Moreover, the present invention described herein may have other embodiments and may be implemented or realized in a variety of ways.

Claims

1. A quantitative evaluation method for the quality of network threat intelligence, characterized in that: include: Acquire a quality assessment field of network threat intelligence, wherein the quality assessment field has a plurality of assessment samples, and the quality assessment field includes: raw data, intelligence, and service providers; Obtain M quality factors corresponding to the quality assessment field, arbitrarily select N of the quality factors as target quality factors, and weight the assessment samples based on the target quality factors to obtain a sorting sequence, and establish a network threat intelligence quality assessment model based on the sorting sequence; wherein, N <M; The network threat intelligence quality assessment model is applied to a preset field, and the network threat intelligence quality assessment model is quantitatively evaluated based on objectivity, subjectivity, performance, behavior and accuracy to obtain a quality assessment score for the preset field.

2. According to claim 1, a method for quantitatively evaluating the quality of network threat intelligence is characterized in that: The process of obtaining M quality factors corresponding to the quality assessment field includes: when the quality assessment field is raw data, the quality factors corresponding to the raw data include at least one of accuracy, timeliness, completeness, consistency, relevance, feasibility and value; when the quality assessment field is intelligence data, the quality factors corresponding to the intelligence data include at least one of accuracy, clarity, availability, relevance, timeliness, completeness, digestibility and credibility; when the quality assessment field is a service provider, the quality factors corresponding to the service provider include at least one of user evaluation, total quality of shared information, traceability, provability and service price.

3. According to claim 1, a method for quantitatively evaluating the quality of network threat intelligence is characterized in that: The process of weighting and sorting the evaluation samples to obtain a sorting sequence includes: After obtaining the relative importance of each target quality factor, a score calculation is performed to obtain a pair comparison matrix, and all elements in the pair comparison matrix are subjected to standard normalization processing to obtain a standardized pair comparison matrix; Calculating a single ranking weight of the standardized pairwise comparison matrix, and calculating a maximum eigenvalue of the standardized pairwise comparison matrix according to the single ranking weight; Calculating a consistency ratio according to the maximum eigenvalue, and dynamically adjusting all elements in the standardized pairwise comparison matrix according to the consistency ratio to obtain an updated standardized pairwise comparison matrix; The single ranking weight of each row element in the updated standardized pairwise comparison matrix is ​​calculated, and the total ranking weight is obtained after weighted summation. The total ranking weight is sorted from large to small to obtain the ranking sequence of the evaluation samples corresponding to each quality assessment field.

4. A method for quantitatively evaluating the quality of network threat intelligence according to claim 3, characterized in that: Each element in the pairwise comparison matrix represents the relative importance of the row target quality factor relative to the column target quality factor. The greater the relative importance, the more important the row target quality factor is relative to the column target quality factor.

5. A method for quantitatively evaluating the quality of network threat intelligence according to claim 3, characterized in that: In the process of dynamically adjusting all elements in the standardized pairwise comparison matrix according to the consistency ratio, a convergence threshold needs to be set for consistency judgment. When the consistency ratio is greater than the convergence threshold, the element values ​​in the pairwise comparison matrix need to be changed again until the consistency ratio is less than the convergence threshold.

6. A method for quantitatively evaluating the quality of network threat intelligence according to claim 1, characterized in that: The preset domain is an industrial domain, which includes an industrial domain set, information technology assets, and unstructured products, wherein the organization operates on the industrial domain set.

7. A method for quantitatively evaluating the quality of network threat intelligence according to claim 1, characterized in that: The process of quantitatively evaluating the cyber threat intelligence quality assessment model based on objectivity, subjectivity, performance, behavior, and accuracy includes: A first document similarity function is obtained based on the cosine value of the information technology asset and the unstructured network threat intelligence product, and a second document similarity function is obtained based on the Jaccard similarity of the industrial domain set and the unstructured network threat intelligence product; Obtaining an industrial domain function based on the relative independence of the first document similarity function and the second document similarity function, and obtaining an accuracy score based on the algorithmic determinism of the industrial domain function; Obtaining a performance score based on the time complexity of the first document similarity function and the second document similarity function; Based on the subjectivity and objectivity of the data of the industrial domain function and the subjectivity and objectivity of the measurement method, the subjective and objective scores are obtained respectively, and the behavioral score is obtained after sensitivity analysis of the industrial domain function.

8. A method for quantitatively evaluating the quality of network threat intelligence according to claim 7, characterized in that: In the process of obtaining the performance score, the comprehensive time complexity is obtained based on the time complexity of the first document similarity function and the second document similarity function, and the performance score is obtained based on the comprehensive time complexity.

9. A method for quantitatively evaluating the quality of network threat intelligence according to claim 8, characterized in that: The comprehensive time complexity is the product of the time complexity of the first document similarity function and the second document similarity function.

Images