A network threat intelligence quality quantitative evaluation method
By establishing a network threat intelligence quality assessment model and utilizing quality factors and quantitative assessment methods from multiple assessment samples, the problems of accuracy and comprehensiveness of assessment results in existing technologies have been solved, achieving a more objective and comprehensive assessment effect.
Patent Information
- Application Number
- CN202411090842.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2025-12-19
- Estimated Expiration
- 2044-08-09
AI Technical Summary
Existing methods for assessing cyber threat intelligence are highly subjective and fail to comprehensively cover multiple important dimensions of threat intelligence quality, resulting in low accuracy and comprehensiveness of assessment results.
We employ quality factors from multiple evaluation samples and establish a network threat intelligence quality assessment model by weighting and ranking them. We combine objectivity, subjectivity, performance, behavior, and accuracy for quantitative evaluation, transforming expert subjective judgments into quantifiable values and clearly defining and quantifying evaluation indicators.
This improves the objectivity and comprehensiveness of cyber threat intelligence quality assessment, reduces the influence of subjective judgment, and ensures the accuracy and comprehensiveness of assessment results.
Smart Images
Figure CN119728139B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of network security, and in particular to a network threat intelligence quality quantitative evaluation method. BACKGROUND
[0002] With the increasing complexity and diversity of network security threats, network threat intelligence has become an indispensable part of organizational defense strategies. Network threat intelligence can come from various channels, such as public sources, secret sources, social media, and honeypot traps, which leads to the increasing diversity and massiveness of network threat intelligence sources, thereby challenging the effective evaluation of network threat intelligence quality. Therefore, how to quantitatively evaluate the quality of network threat intelligence is of great significance.
[0003] Current network threat intelligence evaluation methods generally use means such as crawlers and API interfaces to automatically collect relevant network threat intelligence data, fuse network threat intelligence data from different sources, use predefined rules to preliminarily filter and evaluate the fused data, or directly rely on experts' subjective experience and predefined rules to judge the reliability and relevance of relevant network threat intelligence data, in order to improve the processing speed of network threat intelligence, which has certain application value in practice.
[0004] However, the existing method is often highly subjective in scoring, cannot comprehensively cover multiple important dimensions of threat intelligence quality, and has large human errors, which to some extent affects the accuracy and comprehensiveness of the network threat intelligence quality evaluation results. Therefore, there is an urgent need to provide a solution to improve the above problems. SUMMARY
[0005] The purpose of the present application is to provide a network threat intelligence quality quantitative evaluation method to improve the problem of low comprehensiveness and accuracy in network threat intelligence quality evaluation due to high human subjectivity in evaluating network threat intelligence quality by existing methods.
[0006] The network threat intelligence quality quantitative evaluation method provided by the present application adopts the following technical solution:
[0007] In the quality evaluation field of network threat intelligence, the quality evaluation field has multiple evaluation samples, and the quality evaluation field includes original data, intelligence, and service providers;
[0008] M quality factors corresponding to the quality evaluation field are obtained, N quality factors are selected as target quality factors, the evaluation samples are weighted and sorted based on the target quality factors to obtain a sorting sequence, and a network threat intelligence quality evaluation model is established based on the sorting sequence; wherein N
[0009] The network threat intelligence quality evaluation model is applied to a preset field, and after quantitative evaluation of the network threat intelligence quality evaluation model based on objectivity, subjectivity, performance, behavior and accuracy, a quality evaluation score of the preset field is obtained.
[0010] The network threat intelligence quality quantitative evaluation method has the beneficial effects that firstly, the subjective judgment of experts is converted into quantifiable numerical values, reducing the influence of subjective judgment in the network threat intelligence quality evaluation process, thereby improving the objectivity of the evaluation result; secondly, the evaluation indexes of threat intelligence are clearly defined and quantified, improving the problem of lack of clear standards in the traditional evaluation method. The application establishes a comprehensive quantitative evaluation standard and applies the evaluation standard to a preset industrial field, ensuring the comprehensiveness and accuracy of the network threat intelligence quality evaluation.
[0011] Optionally, the process of obtaining M quality factors corresponding to the quality evaluation field comprises: when the quality evaluation field is original data, the quality factors corresponding to the original data include at least one of accuracy, timeliness, integrity, consistency, relevance, feasibility and value; when the quality evaluation field is intelligence data, the quality factors corresponding to the intelligence data include at least one of accuracy, clarity, usability, relevance, timeliness, integrity, digestibility and credibility; when the quality evaluation field is a service provider, the quality factors corresponding to the service provider include at least one of user evaluation, total quality of shared information, traceability, provability and service price.
[0012] Optionally, the process of weighting and sequencing the evaluation samples to obtain a sequencing sequence comprises:
[0013] The process of weighting and sequencing the evaluation samples to obtain a sequencing sequence comprises:
[0014] After obtaining the relative importance of each target quality factor, the pair-wise comparison matrix is obtained after scoring calculation, and all elements in the pair-wise comparison matrix are subjected to standard normalization processing to obtain a standardized pair-wise comparison matrix.
[0015] The single sequencing weight of the standardized pair-wise comparison matrix is calculated, and the maximum eigenvalue of the standardized pair-wise comparison matrix is calculated according to the single sequencing weight.
[0016] The consistency ratio is calculated according to the maximum eigenvalue, and all elements in the standardized pair-wise comparison matrix are dynamically adjusted according to the consistency ratio to obtain an updated standardized pair-wise comparison matrix.
[0017] The single ranking weight of each element in the updated normalized pairwise comparison matrix is calculated, and the total ranking weight is obtained by weighted summation, and the ranking sequence of the evaluation sample corresponding to each quality evaluation field is obtained by sorting the total ranking weight from large to small.
[0018] Optionally, each element in the pairwise comparison matrix represents the relative importance of the row target quality factor with respect to the column target quality factor, and the greater the relative importance, the more important the row target quality factor with respect to the column target quality factor.
[0019] Optionally, during the process of dynamically adjusting all elements in the normalized pairwise comparison matrix according to the consistency ratio, a convergence threshold is set for consistency judgment, and when the consistency ratio is greater than the convergence threshold, the element value in the pairwise comparison matrix needs to be changed again until the consistency ratio is less than the convergence threshold.
[0020] Optionally, the preset field is an industrial field, and the industrial field includes an industrial domain set, information technology assets, and unstructured products, wherein the organization runs on the industrial domain set.
[0021] Optionally, the process of quantitatively evaluating the network threat intelligence quality evaluation model based on objectivity, subjectivity, performance, behavior, and accuracy includes:
[0022] The first document similarity function is obtained based on the cosine value of the information technology assets and the unstructured network threat intelligence product, and the second document similarity function is obtained based on the Jaccard similarity of the industrial domain set and the unstructured network threat intelligence product.
[0023] The industrial domain function is obtained based on the relative independence of the first document similarity function and the second document similarity function, and the accuracy score is obtained based on the algorithmic determinacy of the industrial domain function;
[0024] The performance score is obtained based on the time complexity of the first document similarity function and the second document similarity function.
[0025] The subjective score and the objective score are obtained based on the data subjectivity and objectivity of the industrial domain function and the subjectivity and objectivity of the measurement method, respectively, and the behavior score is obtained after sensitivity analysis of the industrial domain function.
[0026] Optionally, during the process of obtaining the performance score, the comprehensive time complexity is obtained based on the time complexity of the first document similarity function and the second document similarity function, and the performance score is obtained based on the comprehensive time complexity.
[0027] Optionally, the comprehensive time complexity is the product of the time complexity of the first document similarity function and the second document similarity function. BRIEF DESCRIPTION OF DRAWINGS
[0028] Figure 1 A flow chart of a network threat intelligence quality quantitative evaluation method provided by the present application is shown in the figure.
[0029] Figure 2 A threat intelligence quality evaluation index system provided by the present application is shown in the figure. DETAILED DESCRIPTION
[0030] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application. Unless otherwise defined, the technical terms or scientific terms used herein should be understood as the common meanings thereof by those of ordinary skill in the art. The "comprising" and similar words used in the present application mean that the elements or objects before the word encompass the elements or objects listed after the word and their equivalents, and do not exclude other elements or objects.
[0031] In some embodiments, referring to Figure 1 A flow chart of a network threat intelligence quality quantitative evaluation method provided by the present application is shown in the figure, which includes the following steps:
[0032] S1, obtaining a quality evaluation field of network threat intelligence, the quality evaluation field having a plurality of evaluation samples, the quality evaluation field including original data, intelligence and service providers;
[0033] S2, obtaining M quality factors corresponding to the quality evaluation field, selecting N quality factors as target quality factors, and performing weight ordering on the evaluation samples based on the target quality factors to obtain an ordering sequence, and establishing a network threat intelligence quality evaluation model based on the ordering sequence; wherein N
[0034] S3, applying the network threat intelligence quality evaluation model to a preset field, and obtaining a quality evaluation score of the preset field after performing quantitative evaluation on the network threat intelligence quality evaluation model based on objectivity, subjectivity, performance, behavior and accuracy.
[0035] In some embodiments, referring to Figure 2, which indicates a network threat intelligence quality evaluation index system provided by the application, and specifically comprises: first, determining the evaluation field of the network threat intelligence quality to be evaluated, wherein the evaluation field is composed of original data, intelligence, and service providers. Second, different evaluation fields have different quality factors. The original data field has quality factors such as value, accuracy, and integrity. The intelligence field has quality factors such as relevance and timeliness. The service provider has quality factors such as user evaluation, traceability, and service price. The management decision maker can select any one as the quality evaluation field according to the actual demand.
[0036] In some embodiments, in the step S1 of acquiring the quality evaluation field of the network threat intelligence, the quality evaluation field is any one of the original data, intelligence, and service provider, and each quality evaluation field has a plurality of corresponding evaluation samples, specifically including: the original data has a plurality of original data samples, the intelligence has a plurality of intelligence samples, and the service provider has a plurality of service provider samples.
[0037] In some embodiments, in the step S2 of acquiring the M quality factors corresponding to the quality evaluation field, when the quality evaluation field is original data, the quality factors corresponding to the original data include at least one of accuracy, timeliness, integrity, consistency, relevance, feasibility, and value.
[0038] Further, when the quality evaluation field is intelligence data, the quality factors corresponding to the intelligence data include at least one of accuracy, clarity, availability, relevance, timeliness, integrity, digestibility, and credibility.
[0039] Still further, when the quality evaluation field is a service provider, the quality factors corresponding to the service provider include at least one of user evaluation, total quality of shared information, traceability, provability, and service price.
[0040] In some embodiments, the process of performing step S2 to weight and sort the evaluation samples to obtain a sorting sequence comprises:
[0041] S2-1, acquiring the relative importance of each target quality factor, performing scoring calculation to acquire a pair-wise comparison matrix, and performing standard normalization processing on all elements in the pair-wise comparison matrix to obtain a standardized pair-wise comparison matrix.
[0042] S2-2, calculating the single sorting weight of the standardized pair-wise comparison matrix, and calculating the maximum eigenvalue of the standardized pair-wise comparison matrix according to the single sorting weight.
[0043] S2-3, calculating a consistency ratio according to the maximum eigenvalue, and dynamically adjusting all elements in the normalized pairwise comparison matrix according to the consistency ratio to obtain an updated normalized pairwise comparison matrix;
[0044] S2-4, calculating a single ranking weight of each row element in the updated normalized pairwise comparison matrix, and obtaining a total ranking weight by weighted summation, and obtaining a ranking sequence of the evaluation sample corresponding to each quality evaluation field by ranking the total ranking weight from large to small.
[0045] Specifically, when constructing the pairwise comparison matrix in step S2-1, each element in the pairwise comparison matrix represents the relative importance of the row target quality factor with respect to the column target quality factor. The greater the relative importance, the more important the row target quality factor with respect to the column target quality factor. The relative importance is valued in 1 to 9.
[0046] In fact, a relative importance of 1 indicates that the row target quality factor and the column target quality factor are equally important, and a relative importance of 9 indicates that the row target quality factor is extremely important compared to the column target quality factor. Since it is a pairwise comparison, the importance of the row target quality factor i with respect to the column target quality factor j is X ij , and the importance of the column target quality factor j with respect to the row target quality factor i is
[0047] Further, the process of performing step S2-1 to standardize and normalize all elements in the pairwise comparison matrix to obtain the normalized pairwise comparison matrix includes:
[0048] Obtaining the sum of each column element in the pairwise comparison matrix, and dividing each element by the sum of the column to obtain a normalized pairwise comparison matrix with a sum of 1 in each column. The value of the element in the normalized pairwise comparison matrix is:
[0049]
[0050] wherein r ij represents the value of the element in the normalized pairwise comparison matrix, represents the sum of each column element in the pairwise comparison matrix, n is the order of the pairwise comparison matrix, and x ij represents the value of the element in the pairwise comparison matrix.
[0051] Further, when performing step S2-2 to calculate the single ranking weight of the normalized pairwise comparison matrix, the single ranking weight is the average value of each row element in the normalized pairwise comparison matrix, which is represented as:
[0052]
[0053] wherein ωi is a single ranking weight.
[0054] Further, in the step S2-2, the single ranking weight ω i In the calculation of the maximum eigenvalue of the normalized pairwise comparison matrix, the maximum eigenvalue λ max is expressed as:
[0055]
[0056] where (Xω) i i i i is the i-th eigenvalue of the normalized pairwise comparison matrix.
[0057] Further, in the step S2-3, the process of dynamically adjusting all elements in the normalized pairwise comparison matrix according to the consistency ratio includes:
[0058] The consistency degree measurement index CI of the pairwise comparison matrix is obtained, and the mathematical expression is:
[0059]
[0060] where λ max is the maximum eigenvalue of the pairwise comparison matrix X.
[0061] The random consistency index RI is obtained according to the order of the pairwise comparison matrix. In fact, RI is a preset index, which depends on the order of the matrix. It is the average CI value calculated from a large number of randomly filled pairwise comparison matrices. Different order matrices have different RI values.
[0062] The consistency ratio CR is obtained based on the consistency degree measurement index CI and the random consistency index RI, and the consistency ratio CR is expressed as:
[0063]
[0064] In some embodiments, in the step S2-3, in the process of dynamically adjusting all elements in the normalized pairwise comparison matrix according to the consistency ratio, a convergence threshold is set for consistency judgment. When the consistency ratio is greater than the convergence threshold, the element value in the pairwise comparison matrix needs to be changed again until the consistency ratio is less than the convergence threshold.
[0065] Specifically, the convergence threshold is 0.1. When the consistency ratio CR is less than 0.1, it indicates that the consistency of the pairwise comparison matrix can be accepted. When the consistency ratio CR is greater than or equal to 0.1, the elements in the pairwise comparison matrix need to be adjusted again until the consistency ratio CR is less than 0.1.
[0066] Further, when the total ranking weight is obtained by weighted summation in the execution of step S2-4, the expression of the total ranking weight is:
[0067]
[0068] wherein b i represents the total ranking weight of the i-th quality evaluation field, a ij represents the single ranking weight of the i-th quality evaluation field for the j-th target quality factor, and m represents the number of target quality factors, which is the same as the order n of the pair-wise matrix.
[0069] In fact, the ranking sequence of the evaluation sample corresponding to each quality evaluation field is obtained by ranking the total ranking weight from large to small, and according to the ranking sequence, the security administrator can select the best evaluation sample in each quality evaluation field, i.e., select the first in the corresponding ranking sequence in each quality evaluation field.
[0070] In some embodiments, in the execution of step S3, the preset field is an industrial field, and the industrial field includes an industrial domain set, information technology assets, and network unstructured products, wherein the organization C runs on a limited industrial domain set D = {d1, d2, … d m}.
[0071] Specifically, the network unstructured product P u is an email, a network forum and social media, a blog and a news article, a technical document, a pdf document, etc., and is represented as: P u = {P1, P2, …, P q}, the information technology asset I is a finite sequence, and is represented as: I = {i1, i2, …, i g}, wherein i1, i2, …, i g represent each information technology asset, g represents the number of finite sequences, and q represents the number of network unstructured products.
[0072] In some embodiments, the process of performing step S3 to quantitatively evaluate the network threat intelligence quality evaluation model based on objectivity, subjectivity, performance, behavior, and accuracy includes:
[0073] S3-1, obtaining a first document similarity function based on the cosine value of the information technology asset and the unstructured network threat intelligence product, and obtaining a second document similarity function based on the Jaccard similarity of the industrial domain set and the unstructured network threat intelligence product;
[0074] S3-2, obtaining an industrial domain function based on the relative independence of the first document similarity function and the second document similarity function, and obtaining an accuracy score based on the algorithmic determinacy of the industrial domain function;
[0075] S3-3, obtaining a performance score based on the time complexity of the first document similarity function and the second document similarity function;
[0076] S3-4, obtaining a subjective score and an objective score based on the subjective and objective nature and the subjective and objective nature of the measurement method of the industrial domain function, respectively, and obtaining a behavior score after sensitivity analysis of the industrial domain function.
[0077] Specifically, in the process of performing step S3-1 to obtain the first document similarity function based on the cosine value of the information technology asset and the unstructured network threat intelligence product, the process is as follows:
[0078]
[0079] wherein d I and are the document vector transformations of I and P un , respectively.
[0080] Further, the process of obtaining the second document similarity function based on the Jaccard similarity of the industrial domain set and the unstructured network threat intelligence product is as follows:
[0081]
[0082] wherein d D represents the document vector transformation of the industrial domain set D,
[0083] Further, in the process of performing step S3-2, the industrial domain function is obtained based on the relative independence of the first document similarity function and the second document similarity function, the process is as follows:
[0084] Based on the relative independence of the first document similarity function F1(I, P un ) and the second document similarity function F2(D, P un ), the industrial domain function F(X) is calculated, which is represented as:
[0085]
[0086] wherein the industrial domain function F(X) is obtained by multiplying the first document similarity function F1(I, P un ) and the second document similarity function F2(D, P un ), and F1(I, P un ) is calculated by the cosine function, and F2(D, Pun The result is obtained through the Jaccard similarity function. Since the cosine function and the Jaccard similarity function are deterministic, the data is objective, and the variable D = {d1, d2, ..., d...} m The industrial sector of organization C is determined by human factors, and therefore the measurement method is subjective, that is, it is a subjective measurement of objective data.
[0087] In fact, accuracy is used to describe the predicted value M. c Compared with the actual value M r The degree of agreement is measured by the difference between the predicted and actual values, denoted by b. In assessing the quality of cyber threat intelligence, an accuracy score is defined as A = A(b), where A ∈ [0,1]. This accuracy score converts the deviation b into a value between 0 and 1, where 0 represents complete inaccuracy (maximum deviation) and 1 represents complete accuracy (no deviation). The mathematical relationship between the predicted and actual values is: M r =A(b)M c .
[0088] In an ideal situation, A = A(b) = 1, indicating that there is no bias. In the process of calculating the industrial domain function F(X), F(X) is obtained through a specific mathematical calculation formula, rather than through an experiment with uncertainty. Its acquisition process is deterministic, so no bias is introduced when calculating F(X), and the accuracy score is A = 1.
[0089] Furthermore, during the execution of steps S3-4, in the process of obtaining subjective and objective scores based on the subjectivity and objectivity of data and the subjectivity and objectivity of measurement methods according to industrial domain functions, the quality assessment of network threat intelligence is divided into four subjective / objective levels, as shown in Table 1:
[0090] Table 1 Subjectivity / Objectivity Levels in Network Threat Intelligence Quality Assessment
[0091]
[0092] Furthermore, OO is assigned 4 points, SO is assigned 3 points, OS is assigned 2 points, and SS is assigned 1 point.
[0093] Since the way to obtain the industrial domain function metric F(X) is a subjective measurement of objective data, according to Table 2, the subjective and objective score is 3 points.
[0094] In some embodiments, in the process of obtaining the performance score in step S3-3, a comprehensive time complexity is obtained based on the time complexity of the first document similarity function and the second document similarity function, and the performance score is obtained based on the comprehensive time complexity. Wherein, the time complexity and the corresponding score are shown in Table 2.
[0095] Table 2 Time complexity corresponding score table
[0096]
[0097]
[0098] In fact, the time complexity of the first document similarity function F1(I, P un ) is O(n), and since the logical and intersection operation is avoided when calculating the second document similarity function F2(D, P un ), the performance of F2(D, P un ) is O(1), therefore, the comprehensive time complexity is the product of the time complexity of the first document similarity function and the second document similarity function, denoted as: O(n) × O(1) = O(n), according to Table 2, the performance score P = 3.
[0099] Further, the baseline behavior factor B baseline is represented as:
[0100] B baseline = ω O ·O + ω S ·S + ω P ·P + ω A ·A;
[0101] Wherein, O represents subjectivity, S represents objectivity, P represents performance score, A represents accuracy score, ω O represents the weight of subjectivity, ω S represents the weight of objectivity, ω P represents the weight of performance score, and ω A represents the weight of accuracy score.
[0102] Further, the baseline values of O, S, P, and A are increased by 10% respectively to obtain the updated behavior factor B new,i , represented as:
[0103] B new,i = ω O ·O' + ω S ·S' + ω P ·P' + ω A ·A';
[0104] Wherein, O' represents the updated subjectivity, S' represents the updated objectivity, P' represents the updated performance score, and A' represents the updated accuracy score.
[0105] Further, based on the baseline behavior factor B baseline and the updated behavior factor B new,i The sensitivity index Sensitivity Index is obtained i , which is expressed as:
[0106]
[0107] Further, the score of the behavior factor is the value of the sensitivity index Sensitivity Index i .
[0108] Further, the accuracy score, the performance score, the subjective and objective score, and the behavior score are obtained, i.e., the accuracy score A is 1, the subjective and objective score OS is 3, the performance score P is 3, and the behavior score is the value of the sensitivity index Sensitivity Index i , and the quantitative evaluation process of the network threat intelligence quality is completed.
[0109] Although the embodiments of the present application have been described in detail above, it is obvious to those skilled in the art that various modifications and changes can be made to the embodiments. However, it should be understood that such modifications and changes belong to the scope and spirit of the present application described in the claims. Moreover, the present application described herein can have other embodiments and can be implemented or realized in various ways.
Claims
1. A method for quantitative evaluation of network threat intelligence quality, characterized in that, The method comprises the following steps: Obtaining a quality evaluation field of network threat intelligence, the quality evaluation field having a plurality of evaluation samples, the quality evaluation field comprising original data, intelligence and service providers; Obtaining M quality factors corresponding to the quality evaluation field, selecting N quality factors as target quality factors, and performing weight ordering on the evaluation samples based on the target quality factors to obtain a sorting sequence, and establishing a network threat intelligence quality evaluation model based on the sorting sequence; wherein N < M; the process of performing weight ordering on the evaluation samples to obtain a sorting sequence comprises: obtaining the relative importance of each target quality factor, performing scoring calculation, obtaining a pairwise comparison matrix, and performing standard normalization processing on all elements in the pairwise comparison matrix to obtain a standardized pairwise comparison matrix; calculating the single ordering weight of the standardized pairwise comparison matrix, and calculating the maximum eigenvalue of the standardized pairwise comparison matrix according to the single ordering weight; calculating the consistency ratio according to the maximum eigenvalue, and dynamically adjusting all elements in the standardized pairwise comparison matrix according to the consistency ratio to obtain an updated standardized pairwise comparison matrix; calculating the single ordering weight of each row element in the updated standardized pairwise comparison matrix, and obtaining the total ordering weight by weighted summation; the total ordering weight is sorted from large to small to obtain the sorting sequence of the evaluation samples corresponding to each quality evaluation field; Applying the network threat intelligence quality evaluation model to a preset field, and obtaining the quality evaluation score of the preset field by quantitatively evaluating the network threat intelligence quality evaluation model based on objectivity, subjectivity, performance, behavior and accuracy; wherein the process of quantitatively evaluating the network threat intelligence quality evaluation model based on objectivity, subjectivity, performance, behavior and accuracy comprises: obtaining a first document similarity function based on the cosine value of information technology assets and unstructured network threat intelligence products, obtaining a second document similarity function based on the Jaccard similarity of industrial domain set and unstructured network threat intelligence products; obtaining an industrial domain function based on the relative independence of the first document similarity function and the second document similarity function, and obtaining an accuracy score based on the algorithm determinacy of the industrial domain function; obtaining a performance score based on the time complexity of the first document similarity function and the second document similarity function; obtaining subjective and objective scores based on the data subjectivity and objectivity of the industrial domain function and the subjectivity and objectivity of the measurement method, respectively, and obtaining a behavior score by performing sensitivity analysis on the industrial domain function.
2. The method of claim 1, wherein, The process of obtaining M quality factors corresponding to the quality evaluation field comprises: when the quality evaluation field is original data, the quality factors corresponding to the original data comprise at least one of accuracy, timeliness, integrity, consistency, relevance, feasibility and value; when the quality evaluation field is intelligence data, the quality factors corresponding to the intelligence data comprise at least one of accuracy, clarity, availability, relevance, timeliness, integrity, digestibility and credibility; when the quality evaluation field is a service provider, the quality factors corresponding to the service provider comprise at least one of user evaluation, total quality of shared information, traceability, provability and service price.
3. The method of claim 1, wherein, Each element in the pair-wise comparison matrix represents the relative importance of the row target quality factor with respect to the column target quality factor, and the greater the relative importance, the more important the row target quality factor with respect to the column target quality factor.
4. The method of claim 1, wherein, In the process of dynamically adjusting all elements in the standardized pair-wise comparison matrix according to the consistency ratio, a convergence threshold is set for consistency judgment, and when the consistency ratio is greater than the convergence threshold, the element values in the pair-wise comparison matrix need to be changed again until the consistency ratio is less than the convergence threshold.
5. The method of claim 1, wherein, The preset field is an industrial field, and the industrial field comprises an industrial domain set, information technology assets and unstructured products, wherein the organization runs on the industrial domain set.
6. The method of claim 1, wherein, In the process of obtaining the performance score, a comprehensive time complexity is obtained based on the time complexity of the first document similarity function and the second document similarity function, and the performance score is obtained based on the comprehensive time complexity.
7. The method of claim 6, wherein, The comprehensive time complexity is the product of the time complexity of the first document similarity function and the second document similarity function.