Business model driven vertical domain knowledge graph quality evaluation method

Abstract

The invention discloses a service model driven vertical domain knowledge graph quality evaluation method, which comprises the following steps of: 1, establishing a service model of a target domain for describing and explaining an operation mode and a service execution process of the domain; 2, importing a domain knowledge graph, providing business related knowledge contained in the domain knowledge graph, and carrying out knowledge support and data preparation on a quality evaluation process; 3, determining the quality evaluation dimensions of the domain knowledge graph according to the properties of the domain knowledge graph, and defining the measurement index of each dimension; the quality evaluation dimension comprises an accuracy dimension, a completeness dimension, a consistency dimension and a robustness dimension. 4, establishing a business-driven knowledge graph quality evaluation model, and calculating a measurement index score of each quality evaluation dimension in the domain knowledge graph by using the evaluation model; and 5, simulating the specific task executed by the business model to verify the reliability of the domain knowledge graph quality evaluation result.

Description

Technical Field

[0001] The present invention belongs to the technical field of data quality assessment, and specifically is a vertical field knowledge graph quality assessment method driven by a business model. Background Art

[0002] Knowledge graph is a structured knowledge representation based on graphs, which can model entities, concepts, attributes and the relationships between them in the real world. Vertical domain knowledge graph is a knowledge graph that is modeled and applied to a specific industry or field. Based on the conventional knowledge graph, it further focuses on a vertical field and more deeply describes the entities, concepts, attributes and their semantic relationships in the field. Existing research focuses on the construction of knowledge graphs, including information extraction, knowledge fusion and other construction technologies, but ignores the evaluation of the quality of knowledge graphs.

[0003] Different data sources, construction processes, and update and maintenance methods may affect the quality of the knowledge graph, which may lead to quality problems such as knowledge errors, incompleteness, and inconsistency. For example, the data sets published in the linked open data cloud platform LOD cloud have good quality overall, but there are serious problems in some dimensions. Because users cannot determine whether the knowledge graph meets the quality requirements, these problems may make it difficult to fully utilize the knowledge graph that has been built at a high cost.

[0004] Although the research on traditional data quality assessment has made great progress, there is a lack of technical methods for quality assessment of domain knowledge graphs. In addition, the existing research on quality assessment of domain knowledge graphs is mostly limited to a specific dimension and lacks systematicity. The evaluation results obtained in this way cannot reflect the overall quality of the domain knowledge graph. Therefore, for domain knowledge graphs, based on the existing quality dimensions, building a systematic quality assessment system and effective quality assessment methods will help to fully understand the quality of domain knowledge graphs, so that users can judge whether the domain knowledge graph meets their usage requirements. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a vertical domain knowledge graph quality assessment method based on business model drive, which evaluates the quality of the domain knowledge graph based on business model drive according to the various properties of the domain knowledge graph.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A vertical domain knowledge graph quality assessment method driven by a business model includes the following steps:

[0008] Step 1: Establish a business model for vertical fields

[0009] Establish a business model for the target domain. The business model is used to describe and explain the operation mode and business execution process of the domain. It contains relevant domain knowledge that supports the business and provides data support for the domain knowledge graph.

[0010] Step 2: Import the domain knowledge graph to be evaluated

[0011] Import the domain knowledge graph and provide the business-related knowledge contained in the domain knowledge graph to provide knowledge support and data preparation for the quality assessment process;

[0012] Step 3: Determine the knowledge graph evaluation dimension

[0013] According to the properties of the domain knowledge graph, the quality assessment dimensions of the domain knowledge graph are determined, and the measurement indicators of each dimension are defined; the quality assessment dimensions include accuracy dimension, completeness dimension, consistency dimension and robustness dimension; among which:

[0014] The accuracy dimension is used to describe whether the knowledge in the domain knowledge graph correctly and accurately reflects the objective reality of the domain, including EPV accuracy and ERE accuracy;

[0015] The completeness dimension is used to describe the coverage of domain knowledge by the domain knowledge graph, including four measurement indicators, namely entity class completeness, entity class attribute completeness, entity completeness and entity attribute completeness;

[0016] The consistency dimension is used to describe the degree to which the domain knowledge graph does not have logical contradictions, including entity class consistency and attribute class consistency;

[0017] The robustness dimension is used to describe the degree to which the domain knowledge graph accommodates domain knowledge, including EPV robustness and ERE robustness.

[0018] Step 4: Establish a business-driven knowledge graph quality assessment model

[0019] Establish a business-driven knowledge graph quality assessment model, and use the assessment model to calculate the metric scores of each quality assessment dimension in the domain knowledge graph;

[0020] Step 5: Verify the reliability of knowledge graph quality assessment results

[0021] The reliability of the domain knowledge graph quality assessment results is verified by simulating the execution of specific tasks on the business model.

[0022] Furthermore, in step 3, the measurement indicators of each dimension are:

[0023] (1) The measurement indicators of the accuracy dimension are:

[0024] EPV accuracy is defined as the ratio of correctly described entity-attribute-attribute value triples in the domain knowledge graph to entity-attribute-attribute value triples in the domain knowledge graph, expressed as:

[0025]

[0026] Where: |EPV correct-DKG | is the number of entity-attribute-attribute value triples correctly described in the domain knowledge graph; |EPV DKG | is the number of entity-attribute-attribute value triplets in the domain knowledge graph

[0027] ERE accuracy is defined as the proportion of entity-relationship-entity triples contained in the domain knowledge graph and described correctly to entity-relationship-entity triples in the domain knowledge graph, expressed as:

[0028]

[0029] In the formula: |ERE correct-DKG | is the number of entity-relationship-entity triples correctly described in the domain knowledge graph; |ERE DKG | is the number of entity-relationship-entity triples in the domain knowledge graph.

[0030] (2) The measurement indicators of the completeness dimension are:

[0031] The completeness of entity classes is defined as the proportion of entity classes in the domain knowledge graph to the complete entity classes in the domain, expressed as:

[0032]

[0033] Where: |EC DKG | is the number of entity classes in the domain knowledge graph; |EC complete | is the number of complete entity classes in the field;

[0034] The completeness of entity class attributes is defined as the ratio of attribute classes in the domain knowledge graph to the complete attribute classes in the domain, expressed as:

[0035]

[0036] Where: |PC ec-DKG | is the number of attribute classes belonging to class ec in the domain knowledge graph; |PC ec-complete | is the number of complete attribute classes belonging to class ec in the field.

[0037] Entity completeness is defined as the proportion of entities in the domain knowledge graph to the complete entities in the domain, expressed as:

[0038]

[0039] Where: |E ec-DKG | is the number of entities belonging to class ec in the domain knowledge graph; |E ec-complete | is the number of complete entities belonging to class ec in the field;

[0040] The completeness of entity attributes is defined as the ratio of attributes in the knowledge graph to the complete attributes in the field, expressed as:

[0041]

[0042] Where: |P e-DKG | is the number of entity attributes belonging to entity e in the domain knowledge graph; |P e-complete | is the number of complete attributes belonging to entity e in the domain; |E complete | is the number of complete entities in the field;

[0043] (3) The measurement indicators of the consistency dimension are:

[0044] Entity class consistency is defined as the proportion of correct entity-entity class correspondences in the domain knowledge graph, expressed as:

[0045]

[0046] Where: |E ec-consistent-DKG | is the number of correct entity-entity class correspondences belonging to class ec in the domain knowledge graph;

[0047] Attribute class consistency is defined as the proportion of correct attribute-attribute class correspondences in the domain knowledge graph, expressed as:

[0048]

[0049] Where: |P e-consistent-DKG | is the number of correct attribute-attribute class correspondences belonging to entity e in the domain knowledge graph;

[0050] (4) The measurement indicators of the robustness dimension are:

[0051] EPV robustness is defined as the ratio of entity-attribute-attribute value triples that can be filled into the domain knowledge graph to the complete entity-attribute-attribute value triples in the domain, expressed as:

[0052]

[0053] Where: |EPV refillable | represents the number of entity-attribute-attribute value triples that can be filled into the domain knowledge graph; |EPVcomplete | represents the number of complete entity-attribute-attribute value triplets in the domain knowledge graph;

[0054] ERE robustness is defined as the ratio of entity-relationship-entity triples that can be filled into the domain knowledge graph to the complete entity-relationship-entity triples in the domain, expressed as:

[0055]

[0056] In the formula: |ERE refillable | represents the number of entity-relationship-entity triples that can be filled into the domain knowledge graph; |ERE complete |Represents the number of complete entity-relationship-entity triples in the domain knowledge graph.

[0057] Furthermore, in step 4, the steps of solving the metric scores of each dimension using the business-driven knowledge graph quality assessment model are as follows:

[0058] 41) Execute the business process described in the business model once, obtain and integrate the domain knowledge used in this business process, and count the metric data in the business process knowledge;

[0059] 42) Use the imported domain knowledge graph to query the relevant domain knowledge in this business process, and use the query results to count the relevant measurement indicator data related to the domain knowledge graph;

[0060] 43) Utilize the business-driven metric approximate calculation formula and the evaluation result iteration formula to approximate the quality evaluation result of the domain knowledge graph, and determine whether the quality evaluation result is stable: if so, it indicates that the quality evaluation result is stable, and execute step 44); otherwise, execute step 41);

[0061] 44) Output the domain knowledge graph quality assessment results, including the domain knowledge graph accuracy score Accuracy score , Completeness score , Consistency score and robustness score Robustness score .

[0062] Furthermore, in step 43), the approximate calculation formula of the metric includes:

[0063] (1) Approximate calculation formula for accuracy dimension measurement indicators

[0064] The approximate calculation formula for the EPV accuracy metric is:

[0065]

[0066] Where: |EPV correct-query | represents the number of accurate entity-attribute-attribute value triples in the queried business knowledge; |EPV query | represents the number of entity-attribute-attribute value triplets in the queried business knowledge;

[0067] The approximate calculation formula of the ERE accuracy metric is:

[0068]

[0069] In the formula: |ERE correct-query | represents the number of accurate entity-relationship-entity triples in the queried business knowledge; |ERE query | represents the number of entity-relationship-entity triples in the queried business knowledge;

[0070] (2) Approximate calculation formula for completeness dimension measurement index

[0071] The approximate calculation formula of entity class completeness metric is:

[0072]

[0073] Where: |EC query | indicates the number of entity classes in the queried business knowledge; |EC business | represents the number of entity classes in business knowledge;

[0074] The approximate calculation formula for the entity class attribute completeness metric is:

[0075]

[0076] Where: |PC ec-query | indicates the number of attribute classes contained in each entity class in the queried business knowledge; |PC ec-business |Indicates the number of attribute classes contained in each entity class in business knowledge; |EC business | represents the number of entity classes in business knowledge;

[0077] The approximate calculation formula of entity completeness metric is:

[0078]

[0079] Where: |E ec-query | indicates the number of entities contained in each entity class in the queried business knowledge; |E ec-business | indicates the number of entities contained in each entity in the business knowledge; |EC business | represents the number of entity classes in business knowledge;

[0080] The approximate calculation formula of entity attribute completeness measurement index is:

[0081]

[0082] Where: |P e-query | represents the number of attributes contained in each entity in the queried business knowledge; |P e-business | represents the number of attributes contained in each entity in the business knowledge; |E business | represents the number of entities in business knowledge;

[0083] (3) Approximate calculation formula for consistency dimension measurement indicators

[0084] The approximate calculation formula for the entity class consistency metric is:

[0085]

[0086] Where: |E ec-consistent-query | represents the number of entities with consistent entity class relationships in each entity class in the queried business knowledge; |E ec-query | indicates the number of entities contained in each entity class in the queried business knowledge; |EC business | represents the number of entity classes in business knowledge;

[0087] The approximate calculation formula of the attribute consistency metric is:

[0088]

[0089] Where: |P e-consistent-query | indicates the number of attributes with consistent attribute class relationships in each entity in the queried business knowledge; |P e-query | represents the number of attributes contained in each entity in the queried business knowledge; |E business | represents the number of entities in business knowledge;

[0090] (4) Approximate calculation formula for robustness dimension measurement index

[0091] The approximate calculation formula of the EPV robustness metric is:

[0092]

[0093] Where: |EPV query | represents the number of entity-attribute-attribute value triples in the queried business knowledge; |EPV business | represents the number of entity-attribute-attribute value triplets in business knowledge;

[0094] The approximate calculation formula of the ERE robustness metric is:

[0095]

[0096] In the formula: |ERE query | represents the number of entity-relationship-entity triples in the queried business knowledge; |ERE business |Represents the number of entity-relationship-entity triples in business knowledge.

[0097] Further, in step 43), the evaluation result iteration formula includes:

[0098] Iterative formula for accuracy evaluation results:

[0099]

[0100] Where: Accuracy k The evaluation result of the accuracy of the domain knowledge graph calculated after executing the first k businesses;

[0101] Completeness evaluation result iteration formula:

[0102]

[0103] Where: Completeness k The evaluation result of the completeness of the domain knowledge graph calculated after executing the first k businesses;

[0104] Iteration formula for consistency assessment results:

[0105]

[0106] Where: COnsistency k The evaluation result of the consistency of the domain knowledge graph calculated after executing the first k businesses;

[0107] Iterative formula for robustness evaluation results:

[0108]

[0109] Where: Robustness k It is the evaluation result of the robustness of the domain knowledge graph calculated after executing the first k businesses.

[0110] Further, in step 43), the method for determining the stability of the quality assessment result is:

[0111] The method for judging the stability of accuracy quality assessment results is:

[0112] |Accuracy k+1-Accuracy k |≤ε 1

[0113] Where: ε 1 The maximum deviation allowed for the accuracy quality assessment results;

[0114] If the difference between the accuracy of the domain knowledge graph calculated after two consecutive business executions is less than the set maximum deviation ε 1 , it indicates that the accuracy quality assessment results are stable; otherwise, it indicates that the accuracy quality assessment results are unstable;

[0115] The stability judgment method of completeness quality assessment results is:

[0116] |Completeness k+1 -Completeness k |≤ε 2

[0117] Where: ε 2 The maximum deviation allowed for the completeness quality assessment results;

[0118] If the difference in the evaluation results of the domain knowledge graph completeness calculated after two consecutive business executions is less than the set maximum deviation ε 2 , it indicates that the completeness quality assessment result is stable; otherwise, it indicates that the completeness quality assessment result is unstable;

[0119] The method for judging the stability of consistency quality assessment results is:

[0120] |Consistency k+1 -Consistency k |≤ε 3

[0121] Where: ε 3 The maximum deviation allowed for the consistency quality assessment results;

[0122] If the difference between the evaluation results of the domain knowledge graph consistency calculated after two consecutive business executions is less than the set maximum deviation ε 3 , it indicates that the consistency quality assessment result is stable; otherwise, it indicates that the consistency quality assessment result is unstable;

[0123] The stability judgment method of robustness quality assessment results is:

[0124] |Robustness k+1 -Robustness k |≤ε 4

[0125] Where: ε4 The maximum deviation allowed for robustness quality assessment results;

[0126] If the difference in the robustness evaluation results of the domain knowledge graph calculated after two consecutive business executions is less than the set maximum deviation ε 4 , it indicates that the robustness quality assessment result is stable; otherwise, it indicates that the robustness quality assessment result is unstable.

[0127] Furthermore, in step 5, the method steps for verifying the reliability of the domain knowledge graph quality assessment results through business simulation are as follows:

[0128] 51) Receive the specific task processed by this business model, and retrieve the domain knowledge related to the business task from the domain knowledge graph;

[0129] 52) Substitute the retrieved domain knowledge into the business process described by the business model, and calculate the support of the domain knowledge graph for each business process. i ;

[0130] 53) Summarize the calculation results of the support of the domain knowledge graph for the business process, and obtain the support of the domain knowledge graph for the business task;

[0131] 54) The support degree of the domain knowledge graph for the business task is combined with the quality assessment score DKG of the domain knowledge graph score Make a comparison and judge the reliability of the evaluation results.

[0132] Further, in step 52), the support degree Support i The ratio of the domain knowledge related to the business process retrieved from the domain knowledge graph to the domain knowledge required to execute the business process is expressed as:

[0133] Support i =Accuracy S_i +Completeness S_i +Consistency S_i

[0134] Where: Accuracy S_i The accuracy of the retrieved domain knowledge to support the i-th business process; Completeness S_i The completeness of the retrieved domain knowledge to support the i-th business process; Consistency S_i The retrieved domain knowledge supports the consistency of the i-th business process.

[0135] Further, in step 53), the support degree Support is expressed as:

[0136]

[0137] Where: N is the number of business processes in the business model in this field.

[0138] Further, in step 54), the quality assessment comprehensive score DKG score It is expressed as:

[0139] DKG score =Accuracy score +Completeness score +Consistency score +Robustness score

[0140] Where: Accuracy score Indicates the accuracy score of the domain knowledge graph; Completeness score Indicates the completeness score of the domain knowledge graph; Consistency score Represents the consistency score of the domain knowledge graph; Robustness score Represents the robustness score of the domain knowledge graph;

[0141] The method for judging the reliability of the evaluation results is: setting the reliability threshold ε of the evaluation results, if |DKG score -Support|≤ε, it means that the evaluation result of the domain knowledge graph is reliable; if |DKG score -Support|>ε, it means that the evaluation result of the domain knowledge graph is unreliable.

[0142] The beneficial effects of the present invention are:

[0143] This paper investigates various quality dimensions proposed in the quality research work from traditional data to knowledge graphs, and proposes a vertical domain knowledge graph quality assessment method based on business model drive through sorting and analysis, which includes 4 dimensions and 10 evaluation indicators, and establishes a business-driven knowledge graph quality assessment model to calculate the measurement indicator scores of each dimension of the domain knowledge graph, and evaluates the quality of the domain knowledge graph based on the business model drive according to the various properties of the domain knowledge graph; the method of the present invention can comprehensively, efficiently and accurately evaluate the quality of the vertical domain knowledge graph, and provide strong support for building high-quality domain knowledge graphs. BRIEF DESCRIPTION OF THE DRAWINGS

[0144] In order to make the purpose, technical solution and beneficial effects of the present invention clearer, the present invention provides the following drawings for illustration:

[0145] Figure 1 This is a flow chart of the vertical field knowledge graph quality assessment method driven by a business model according to the present invention;

[0146] Figure 2 This is a schematic diagram of the business-driven knowledge graph quality assessment model;

[0147] Figure 3 Flowchart for verifying the reliability of domain knowledge graph quality assessment results for business simulation. DETAILED DESCRIPTION

[0148] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0149] like Figure 1 As shown, this embodiment is based on a business model-driven vertical domain knowledge graph quality assessment method, and includes the following steps.

[0150] Step 1: Establish a business model for vertical fields

[0151] Establish a business model for the target domain. The business model is used to describe and explain the operating mode and business execution process of the domain. It contains relevant domain knowledge that supports the business and provides data support for the domain knowledge graph.

[0152] Step 2: Import the domain knowledge graph to be evaluated

[0153] Import the domain knowledge graph and provide the business-related knowledge contained in the domain knowledge graph to provide knowledge support and data preparation for the quality assessment process.

[0154] Step 3: Determine the knowledge graph evaluation dimension

[0155] According to the properties of the domain knowledge graph, the quality assessment dimensions of the domain knowledge graph are determined, and measurement indicators of each dimension are defined; the quality assessment dimensions include accuracy dimension, completeness dimension, consistency dimension and robustness dimension.

[0156] (1) Metrics of the accuracy dimension

[0157] The accuracy dimension is used to describe whether the knowledge in the domain knowledge graph correctly and accurately reflects the objective reality of the domain. According to the type of knowledge graph triples, it is divided into entity-property-value (EPV) triples and entity-relationship-entity (ERE) triples. In this embodiment, the accuracy dimension includes EPV accuracy and ERE accuracy.

[0158] EPV accuracy is defined as the ratio of correctly described entity-attribute-attribute value triples in the domain knowledge graph to entity-attribute-attribute value triples in the domain knowledge graph, expressed as:

[0159]

[0160] Where: |EPV correct-DKG | is the number of entity-attribute-attribute value triples correctly described in the domain knowledge graph; |EPV DKG | is the number of entity-attribute-attribute value triplets in the domain knowledge graph.

[0161] ERE accuracy is defined as the proportion of entity-relationship-entity triples contained in the domain knowledge graph and described correctly to entity-relationship-entity triples in the domain knowledge graph, expressed as:

[0162]

[0163] In the formula: |ERE correct-DKG | is the number of entity-relationship-entity triples correctly described in the domain knowledge graph; |ERE DKG | is the number of entity-relationship-entity triples in the domain knowledge graph.

[0164] (2) Completeness Dimension Metrics

[0165] The completeness dimension is used to describe the coverage of the domain knowledge graph on the domain knowledge. According to the knowledge entity characteristics of the domain knowledge graph, the measurement indicators of the completeness dimension are designed. In this embodiment, the completeness dimension includes four measurement indicators, namely entity class completeness, entity class attribute completeness, entity completeness and entity attribute completeness.

[0166] The completeness of entity classes is defined as the proportion of entity classes in the domain knowledge graph to the complete entity classes in the domain, expressed as:

[0167]

[0168] Where: |EC DKG | is the number of entity classes in the domain knowledge graph; |EC complete | is the number of complete entity classes in the field.

[0169] The completeness of entity class attributes is defined as the ratio of attribute classes in the domain knowledge graph to the complete attribute classes in the domain, expressed as:

[0170]

[0171] Where: |PC ec-DKG| is the number of attribute classes belonging to class ec in the domain knowledge graph; |PC ec-complete | is the number of complete attribute classes belonging to class ec in the field.

[0172] Entity completeness is defined as the proportion of entities in the domain knowledge graph to the complete entities in the domain, expressed as:

[0173]

[0174] Where: |E ec-DKG | is the number of entities belonging to class ec in the domain knowledge graph; |E ec-complete | is the number of complete entities belonging to class ec in the field.

[0175] The completeness of entity attributes is defined as the ratio of attributes in the knowledge graph to the complete attributes in the field, expressed as:

[0176]

[0177] Where: |P e-DKG | is the number of entity attributes belonging to entity e in the domain knowledge graph; |P e-complete | is the number of complete attributes belonging to entity e in the domain; |E complete | is the number of complete entities in the field.

[0178] (3) Metrics of the consistency dimension

[0179] The consistency dimension is used to describe the degree to which the domain knowledge graph is logically free of contradictions. According to the knowledge graph model layer, there is a definition of entity classes, and for each entity class, there is a definition of corresponding attributes. Therefore, the proportion of the correct correspondence between the two is used to measure the consistency of the knowledge graph. In this embodiment, the consistency dimension includes entity class consistency and attribute class consistency.

[0180] Entity class consistency is defined as the proportion of correct entity-entity class correspondences in the domain knowledge graph, expressed as:

[0181]

[0182] Where: |E ec-consistent-DKG | is the number of correct entity-entity class correspondences belonging to class ec in the domain knowledge graph.

[0183] Attribute class consistency is defined as the proportion of correct attribute-attribute class correspondences in the domain knowledge graph, expressed as:

[0184]

[0185] Where: |P e-consistent-DKG| is the number of correct attribute-attribute class correspondences belonging to entity e in the domain knowledge graph.

[0186] (4) Metrics of the robustness dimension

[0187] The robustness dimension is used to describe the degree to which the domain knowledge graph accommodates domain knowledge. Domain knowledge is expressed in the form of triples, and the number of triples that can be filled into the knowledge graph pattern layer accounts for the total number of triples in all domain knowledge to measure robustness. According to the classification of triple types, the robustness dimension includes EPV robustness and ERE robustness.

[0188] EPV robustness is defined as the ratio of entity-attribute-attribute value triples that can be filled into the domain knowledge graph to the complete entity-attribute-attribute value triples in the domain, expressed as:

[0189]

[0190] Where: |EPV refillable | represents the number of entity-attribute-attribute value triples that can be filled into the domain knowledge graph; |EPV complete | represents the number of complete entity-attribute-attribute value triplets in the domain knowledge graph;

[0191] ERE robustness is defined as the ratio of entity-relationship-entity triples that can be filled into the domain knowledge graph to the complete entity-relationship-entity triples in the domain, expressed as:

[0192]

[0193] In the formula: |ERE refillable | represents the number of entity-relationship-entity triples that can be filled into the domain knowledge graph; |ERE complete |Represents the number of complete entity-relationship-entity triples in the domain knowledge graph.

[0194] Step 4: Establish a business-driven knowledge graph quality assessment model

[0195] Establish a business-driven knowledge graph quality assessment model, and use the assessment model to calculate the metric scores of each quality assessment dimension in the domain knowledge graph.

[0196] In the actual calculation process, it is impossible to directly obtain the form defined in S3 such as |EC complete| and other variables that describe the complete set of domain knowledge information, making it impossible to calculate the scores of metrics for each dimension of the domain knowledge graph. Therefore, a method that can achieve approximate calculation is needed to achieve approximate calculation of metrics. The domain business model expresses the execution process of the business and contains domain-related knowledge. Based on this situation, consider establishing a business-driven knowledge graph quality assessment model, such as Figure 2 shown.

[0197] The business-driven knowledge graph quality assessment model starts with executing a business model in the field. By summarizing and querying the domain knowledge used when executing the business model, the metric values ​​of the domain knowledge supporting the business process in the domain knowledge graph are obtained. Then, the quality assessment results are approximated through the metric approximation calculation formula and the evaluation result iterative calculation formula, and the quality assessment result stability index is used to control whether the quality assessment results meet the output requirements. Specifically, the steps for solving the metric scores of each dimension using the business-driven knowledge graph quality assessment model are as follows:

[0198] 41) Execute the business process described in the business model once, obtain and integrate the domain knowledge used in this business process, and count the measurement indicator data in the business process knowledge.

[0199] In this step, firstly, all business processes of the business model are executed once. During the execution, domain knowledge is generated in the form of text data, semi-structured data or structured data. Then, the domain knowledge obtained after the business process is generated is summarized and integrated to obtain the knowledge triple set T in the field used by the business process. business ; Finally, the obtained domain knowledge triple set T business Perform statistical analysis to obtain the variable data of the measurement indicators in the business process knowledge. The variable data that needs to be counted in the business process knowledge is shown in Table 1.

[0200] Table 1 Variable data that need to be counted in business process knowledge

[0201] variable meaning <![CDATA[|EC business |]]> The number of entity classes in business knowledge <![CDATA[|PC ec-business |]]> The number of attribute classes contained in each entity class in the business knowledge <![CDATA[|E ec-business |]]> The number of entities contained in each entity in the business knowledge <![CDATA[|P e-business |]]> The number of attributes contained in each entity in the business knowledge <![CDATA[|E business |]]> The number of entities in the business knowledge <![CDATA[|EPV business |]]> The number of EPV triples in business knowledge <![CDATA[|ERE business |]]> The number of ERE triples in business knowledge

[0202] 42) Use the imported domain knowledge graph to query the relevant domain knowledge in this business process, and use the query results to count the relevant measurement indicator data related to the domain knowledge graph.

[0203] In this step, we first use the pre-imported domain knowledge graph as support and use the triple set T business Query to obtain the corresponding triple set in the domain knowledge graph; then perform the domain knowledge triple set T DKGPerform statistical analysis to obtain the variable data of the measurement indicators of the domain knowledge graph supporting the business process knowledge. The variable data that needs to be counted for the domain knowledge graph to support the business process knowledge is shown in Table 2.

[0204] Table 2 Variable data that need to be counted for domain knowledge graph to support business process knowledge

[0205] variable meaning <![CDATA[|EPV query |]]> The number of EPV triples in the queried business knowledge <![CDATA[|EPV correct-query |]]> The exact number of EPV triples in the queried business knowledge <![CDATA[|ERE query |]]> The number of ERE triples in the queried business knowledge <![CDATA[|ERE correct-query |]]> The exact number of EPV triples in the queried business knowledge <![CDATA[|EC query |]]> The number of entity classes in the queried business knowledge <![CDATA[|PC ec-query |]]> The number of attribute classes contained in each entity class in the queried business knowledge <![CDATA[|E ec-query |]]> The number of entities contained in each entity class in the queried business knowledge <![CDATA[|P e-query |]]> The number of attributes contained in each entity in the queried business knowledge <![CDATA[|E ec--consistent-query |]]> The number of entities with consistent entity class relationships in each entity class in the queried business knowledge <![CDATA[|P e-consistent-query |]]> The number of attributes with consistent attribute class relationships in each entity in the queried business knowledge

[0206] 43) Use the business-driven metric approximate calculation formula and the evaluation result iteration formula to approximate the quality evaluation result of the domain knowledge graph, and determine whether the quality evaluation result is stable: if so, it indicates that the quality evaluation result is stable, and execute step 44); otherwise, execute step 41).

[0207] (1) Calculate the values ​​of each metric in the four dimensions of the knowledge graph of the domain to be evaluated during the execution of the business model using the variable data obtained from the statistics of step 41) and step 42) and the approximate calculation formula of the metric driven by the business. Specifically, in this embodiment, the approximate calculation formula of the metric includes:

[0208] (1) Approximate calculation formula for accuracy dimension measurement indicators

[0209] The approximate calculation formula for the EPV accuracy metric is:

[0210]

[0211] Where: |EPV correct-query | represents the number of accurate entity-attribute-attribute value triples in the queried business knowledge; |EPV query | represents the number of entity-attribute-attribute value triplets in the queried business knowledge;

[0212] The approximate calculation formula of the ERE accuracy metric is:

[0213]

[0214] In the formula: |ERE correct-query | represents the number of accurate entity-relationship-entity triples in the queried business knowledge; |ERE query | represents the number of entity-relationship-entity triples in the queried business knowledge;

[0215] (2) Approximate calculation formula for completeness dimension measurement index

[0216] The approximate calculation formula of entity class completeness metric is:

[0217]

[0218] Where: |EC query | indicates the number of entity classes in the queried business knowledge; |EC business | represents the number of entity classes in business knowledge;

[0219] The approximate calculation formula for the entity class attribute completeness metric is:

[0220]

[0221] Where: |PC ec-query | indicates the number of attribute classes contained in each entity class in the queried business knowledge; |PC ec-business |Indicates the number of attribute classes contained in each entity class in business knowledge; |EC business | represents the number of entity classes in business knowledge;

[0222] The approximate calculation formula of entity completeness metric is:

[0223]

[0224] Where: |E ec-query | indicates the number of entities contained in each entity class in the queried business knowledge; |E ec-business | indicates the number of entities contained in each entity in the business knowledge; |EC business | represents the number of entity classes in business knowledge;

[0225] The approximate calculation formula of entity attribute completeness measurement index is:

[0226]

[0227] Where: |P e-query | represents the number of attributes contained in each entity in the queried business knowledge; |P e-business | represents the number of attributes contained in each entity in the business knowledge; |E business | represents the number of entities in business knowledge;

[0228] (3) Approximate calculation formula for consistency dimension measurement indicators

[0229] The approximate calculation formula for the entity class consistency metric is:

[0230]

[0231] Where: |E ec-consistent-query | represents the number of entities with consistent entity class relationships in each entity class in the queried business knowledge; |E ec-query | indicates the number of entities contained in each entity class in the queried business knowledge; |ECbusiness | represents the number of entity classes in business knowledge;

[0232] The approximate calculation formula of the attribute consistency metric is:

[0233]

[0234] Where: |P e-consistent-query | indicates the number of attributes with consistent attribute class relationships in each entity in the queried business knowledge; |P e-query | represents the number of attributes contained in each entity in the queried business knowledge; |E business | represents the number of entities in business knowledge;

[0235] (4) Approximate calculation formula for robustness dimension measurement index

[0236] The approximate calculation formula of the EPV robustness metric is:

[0237]

[0238] Where: |EPV query | represents the number of entity-attribute-attribute value triples in the queried business knowledge; |EPV business | represents the number of entity-attribute-attribute value triplets in business knowledge;

[0239] The approximate calculation formula of the ERE robustness metric is:

[0240]

[0241] In the formula: |ERE query | represents the number of entity-relationship-entity triples in the queried business knowledge; |ERE business |Represents the number of entity-relationship-entity triples in business knowledge.

[0242] Substitute the approximate calculation results of the metric into the iterative calculation formula of the evaluation result to obtain the updated value of the quality evaluation result of the domain knowledge graph after executing this business process. In this embodiment, the iterative formula of the evaluation result includes:

[0243] Iterative formula for accuracy evaluation results:

[0244]

[0245] Where: Accuracy k The evaluation result of the accuracy of the domain knowledge graph calculated after executing the first k businesses;

[0246] Completeness evaluation result iteration formula:

[0247]

[0248] Where: Completeness k The evaluation result of the completeness of the domain knowledge graph calculated after executing the first k businesses;

[0249] Iteration formula for consistency assessment results:

[0250]

[0251] Where: Consistency k The evaluation result of the consistency of the domain knowledge graph calculated after executing the first k businesses;

[0252] Iterative formula for robustness evaluation results:

[0253]

[0254] Where: Robustness k It is the evaluation result of the robustness of the domain knowledge graph calculated after executing the first k businesses.

[0255] In this embodiment, the method for determining the stability of the quality assessment result is:

[0256] The method for judging the stability of accuracy quality assessment results is:

[0257] |Accuracy k+1 -Accuracy k |≤ε 1

[0258] Where: ε 1 The maximum deviation allowed for the accuracy quality assessment results;

[0259] If the difference between the accuracy of the domain knowledge graph calculated after two consecutive business executions is less than the set maximum deviation ε 1 , it indicates that the accuracy quality assessment results are stable; otherwise, it indicates that the accuracy quality assessment results are unstable;

[0260] The stability judgment method of completeness quality assessment results is:

[0261] |Completeness k+1 -Completeness k |≤ε 2

[0262] Where: ε 2 The maximum deviation allowed for the completeness quality assessment results;

[0263] If the difference in the evaluation results of the domain knowledge graph completeness calculated after two consecutive business executions is less than the set maximum deviation ε 2 , it indicates that the completeness quality assessment result is stable; otherwise, it indicates that the completeness quality assessment result is unstable;

[0264] The method for judging the stability of consistency quality assessment results is:

[0265] |Consistency k+1 -Consistency k |≤ε 3

[0266] Where: ε 3 The maximum deviation allowed for the consistency quality assessment results;

[0267] If the difference between the evaluation results of the domain knowledge graph consistency calculated after two consecutive business executions is less than the set maximum deviation ε 3 , it indicates that the consistency quality assessment result is stable; otherwise, it indicates that the consistency quality assessment result is unstable;

[0268] The stability judgment method of robustness quality assessment results is:

[0269] |Robustness k+1 -Robustness k |≤ε 4

[0270] Where: ε 4 The maximum deviation allowed for robustness quality assessment results;

[0271] If the difference in the robustness evaluation results of the domain knowledge graph calculated after two consecutive business executions is less than the set maximum deviation ε 4 , it indicates that the robustness quality assessment result is stable; otherwise, it indicates that the robustness quality assessment result is unstable.

[0272] 44) Output the domain knowledge graph quality assessment results, including the domain knowledge graph accuracy score Accuracy score , Completeness score , Consistency score and robustness score Robustness score .

[0273] Step 5: Verify the reliability of knowledge graph quality assessment results

[0274] The reliability of the domain knowledge graph quality assessment results is verified by simulating the execution of specific tasks on the business model.

[0275] Specifically, Figure 3 As shown in the figure, the method steps for verifying the reliability of the domain knowledge graph quality assessment results through business simulation are as follows:

[0276] 51) Receive the specific task processed by this business model, and retrieve the domain knowledge related to the business task from the domain knowledge graph.

[0277] 52) Substitute the retrieved domain knowledge into the business process described by the business model, and calculate the support of the domain knowledge graph for each business process. i .

[0278] Specific, support i The ratio of the domain knowledge related to the business process retrieved from the domain knowledge graph to the domain knowledge required to execute the business process is expressed as:

[0279] Support i =Accuracy S_i +Completeness S_i +Consistency S_i

[0280] Where: Accuracy S_i The accuracy of the retrieved domain knowledge to support the i-th business process; Completeness S_i The completeness of the retrieved domain knowledge to support the i-th business process; Consistency S_i The retrieved domain knowledge supports the consistency of the i-th business process.

[0281] 53) Summarize the calculation results of the support degree of the domain knowledge graph for the business process, and obtain the support degree Support of the domain knowledge graph for the business task.

[0282] In this embodiment, the support degree Support is expressed as:

[0283]

[0284] Where: N is the number of business processes in the business model in this field.

[0285] 54) The support degree of the domain knowledge graph for the business task is combined with the quality assessment score DKG of the domain knowledge graph score Make a comparison and judge the reliability of the evaluation results.

[0286] In this embodiment, the quality assessment comprehensive score DKG score It is expressed as:

[0287] DKG score =Accuracy score +Completeness score +Consistency score +Robustness score

[0288] Where: Accuracy score Indicates the accuracy score of the domain knowledge graph; Completeness score Indicates the completeness score of the domain knowledge graph; Consistency score Represents the consistency score of the domain knowledge graph; Robustness score Represents the robustness score of the domain knowledge graph;

[0289] The method for judging the reliability of the evaluation results is: setting the reliability threshold ε of the evaluation results, if |DKG score -Support|≤ε, it means that the evaluation result of the domain knowledge graph is reliable; if |DKG score -Support|>ε, it means that the evaluation result of the domain knowledge graph is unreliable.

[0290] The above-described embodiments are only preferred embodiments for fully illustrating the present invention, and the protection scope of the present invention is not limited thereto. Equivalent substitutions or changes made by those skilled in the art based on the present invention are within the protection scope of the present invention. The protection scope of the present invention shall be subject to the claims.

Claims

1. A vertical domain knowledge graph quality assessment method driven by business model, characterized by: The steps include: Step 1: Establish a business model for vertical fields Establish a business model for the target domain. The business model is used to describe and explain the operation mode and business execution process of the domain. It contains relevant domain knowledge that supports the business and provides data support for the domain knowledge graph. Step 2: Import the domain knowledge graph to be evaluated Import the domain knowledge graph and provide the business-related knowledge contained in the domain knowledge graph to provide knowledge support and data preparation for the quality assessment process; Step 3: Determine the knowledge graph evaluation dimension According to the properties of the domain knowledge graph, the quality assessment dimensions of the domain knowledge graph are determined, and the measurement indicators of each dimension are defined; the quality assessment dimensions include accuracy dimension, completeness dimension, consistency dimension and robustness dimension; among which: The accuracy dimension is used to describe whether the knowledge in the domain knowledge graph correctly and accurately reflects the objective reality of the domain, including EPV accuracy and ERE accuracy; The completeness dimension is used to describe the coverage of domain knowledge by the domain knowledge graph, including four measurement indicators, namely entity class completeness, entity class attribute completeness, entity completeness and entity attribute completeness; The consistency dimension is used to describe the degree to which the domain knowledge graph does not have logical contradictions, including entity class consistency and attribute class consistency; The robustness dimension is used to describe the degree to which the domain knowledge graph accommodates domain knowledge, including EPV robustness and ERE robustness; Step 4: Establish a business-driven knowledge graph quality assessment model Establish a business-driven knowledge graph quality assessment model, and use the assessment model to calculate the metric scores of each quality assessment dimension in the domain knowledge graph; Step 5: Verify the reliability of knowledge graph quality assessment results The reliability of the domain knowledge graph quality assessment results is verified by simulating the execution of specific tasks on the business model.

2. The vertical domain knowledge graph quality assessment method based on business model drive according to claim 1 is characterized by: In step 3, the measurement indicators of each dimension are: (1) The measurement indicators of the accuracy dimension are: EPV accuracy is defined as the ratio of correctly described entity-attribute-attribute value triples in the domain knowledge graph to entity-attribute-attribute value triples in the domain knowledge graph, expressed as: Where: |EPV correct―DKG | is the number of entity-attribute-attribute value triples correctly described in the domain knowledge graph; |EPV DKG | is the number of entity-attribute-attribute value triplets in the domain knowledge graph ERE accuracy is defined as the proportion of entity-relationship-entity triples contained in the domain knowledge graph and described correctly to entity-relationship-entity triples in the domain knowledge graph, expressed as: In the formula: |ERE correct―DKG | is the number of entity-relationship-entity triples correctly described in the domain knowledge graph; |ERE DKG | is the number of entity-relationship-entity triples in the domain knowledge graph; (2) The measurement indicators of the completeness dimension are: The completeness of entity classes is defined as the proportion of entity classes in the domain knowledge graph to the complete entity classes in the domain, expressed as: Where: |EC DKG | is the number of entity classes in the domain knowledge graph; |EC complete | is the number of complete entity classes in the field; The completeness of entity class attributes is defined as the ratio of attribute classes in the domain knowledge graph to the complete attribute classes in the domain, expressed as: Where: |PC ec―DKG | is the number of attribute classes belonging to class ec in the domain knowledge graph; |PC ec―complete | is the number of complete attribute classes belonging to class ec in the field; Entity completeness is defined as the proportion of entities in the domain knowledge graph to the complete entities in the domain, expressed as: Where: |E ec―DKG | is the number of entities belonging to class ec in the domain knowledge graph; |E ec―complete | is the number of complete entities belonging to class ec in the field; The completeness of entity attributes is defined as the ratio of attributes in the knowledge graph to the complete attributes in the field, expressed as: Where: |P e―DKG | is the number of entity attributes belonging to entity e in the domain knowledge graph; |P e―complete | is the number of complete attributes belonging to entity e in the domain; |E complete | is the number of complete entities in the field; (3) The measurement indicators of the consistency dimension are: Entity class consistency is defined as the proportion of correct entity-entity class correspondences in the domain knowledge graph, expressed as: Where: |E ec―consistent―DKG | is the number of correct entity-entity class correspondences belonging to class ec in the domain knowledge graph; Attribute class consistency is defined as the proportion of correct attribute-attribute class correspondences in the domain knowledge graph, expressed as: Where: |P e―consistent―DKG | is the number of correct attribute-attribute class correspondences belonging to entity e in the domain knowledge graph; (4) The measurement indicators of the robustness dimension are: EPV robustness is defined as the ratio of entity-attribute-attribute value triples that can be filled into the domain knowledge graph to the complete entity-attribute-attribute value triples in the domain, expressed as: Where: |EPV refilable | represents the number of entity-attribute-attribute value triples that can be filled into the domain knowledge graph; |EPV complete | represents the number of complete entity-attribute-attribute value triplets in the domain knowledge graph; ERE robustness is defined as the ratio of entity-relationship-entity triples that can be filled into the domain knowledge graph to the complete entity-relationship-entity triples in the domain, expressed as: In the formula: |ERE refillable | represents the number of entity-relationship-entity triples that can be filled into the domain knowledge graph; |ERE complete |Represents the number of complete entity-relationship-entity triples in the domain knowledge graph.

3. The vertical domain knowledge graph quality assessment method based on business model drive according to claim 1 is characterized by: In step 4, the steps of solving the metric scores of each dimension using the business-driven knowledge graph quality assessment model are as follows: 41) Execute the business process described in the business model once, obtain and integrate the domain knowledge used in this business process, and count the metric data in the business process knowledge; 42) Use the imported domain knowledge graph to query the relevant domain knowledge in this business process, and use the query results to count the relevant measurement indicator data related to the domain knowledge graph; 43) Utilize the business-driven metric approximate calculation formula and the evaluation result iteration formula to approximate the quality evaluation result of the domain knowledge graph, and determine whether the quality evaluation result is stable: if so, it indicates that the quality evaluation result is stable, and execute step 44); otherwise, execute step 41); 44) Output the domain knowledge graph quality assessment results, including the domain knowledge graph accuracy score Accuracy score , Consistency score , Consistency score and robustness score Robustness score .

4. The vertical domain knowledge graph quality assessment method based on business model drive according to claim 3 is characterized by: In step 43), the approximate calculation formula of the metric includes: (1) Approximate calculation formula for accuracy dimension measurement indicators The approximate calculation formula for the EPV accuracy metric is: Where: |EPV correct―query | represents the number of accurate entity-attribute-attribute value triples in the queried business knowledge; |EPV query | represents the number of entity-attribute-attribute value triplets in the queried business knowledge; The approximate calculation formula of the ERE accuracy metric is: In the formula: |ERE correct―query | represents the number of accurate entity-relationship-entity triples in the queried business knowledge; |ERE query | represents the number of entity-relationship-entity triples in the queried business knowledge; (2) Approximate calculation formula for completeness dimension measurement index The approximate calculation formula of entity class completeness metric is: Where: |EC query | indicates the number of entity classes in the queried business knowledge; |EC business | represents the number of entity classes in business knowledge; The approximate calculation formula for the entity class attribute completeness metric is: Where: |PC ec―query | indicates the number of attribute classes contained in each entity class in the queried business knowledge; |PC ce―business |Indicates the number of attribute classes contained in each entity class in business knowledge; |EC business | represents the number of entity classes in business knowledge; The approximate calculation formula of entity completeness metric is: Where: |E ec―query | indicates the number of entities contained in each entity class in the queried business knowledge; |E ec―business | indicates the number of entities contained in each entity in the business knowledge; |EC business | represents the number of entity classes in business knowledge; The approximate calculation formula of entity attribute completeness measurement index is: Where: |P e―query | represents the number of attributes contained in each entity in the queried business knowledge; |P e―business | represents the number of attributes contained in each entity in the business knowledge; |E business | represents the number of entities in business knowledge; (3) Approximate calculation formula for consistency dimension measurement indicators The approximate calculation formula for the entity class consistency metric is: Where: |E ec―consistent―query | represents the number of entities with consistent entity class relationships in each entity class in the queried business knowledge; |E ec―query | indicates the number of entities contained in each entity class in the queried business knowledge; |EC business | represents the number of entity classes in business knowledge; The approximate calculation formula of the attribute consistency metric is: Where: |P e―consistent―query | indicates the number of attributes with consistent attribute class relationships in each entity in the queried business knowledge; |P e―query | represents the number of attributes contained in each entity in the queried business knowledge; |E business | represents the number of entities in business knowledge; (4) Approximate calculation formula for robustness dimension measurement index The approximate calculation formula of the EPV robustness metric is: Where: |EPV query | represents the number of entity-attribute-attribute value triples in the queried business knowledge; |EPV business | represents the number of entity-attribute-attribute value triplets in business knowledge; The approximate calculation formula of the ERE robustness metric is: In the formula: |ERE query | represents the number of entity-relationship-entity triples in the queried business knowledge; |ERE business |Represents the number of entity-relationship-entity triples in business knowledge.

5. The vertical domain knowledge graph quality assessment method based on business model drive according to claim 3 is characterized by: In step 43), the evaluation result iteration formula includes: Iterative formula for accuracy evaluation results: Where: Accuracy k The evaluation result of the accuracy of the domain knowledge graph calculated after executing the first k businesses; Completeness evaluation result iteration formula: Where: Completeness k The evaluation result of the completeness of the domain knowledge graph calculated after executing the first k businesses; Iteration formula for consistency assessment results: Where: Consistency k The evaluation result of the consistency of the domain knowledge graph calculated after executing the first k businesses; Iterative formula for robustness evaluation results: Where: Robustness k It is the evaluation result of the robustness of the domain knowledge graph calculated after executing the first k businesses.

6. The vertical domain knowledge graph quality assessment method based on business model drive according to claim 3 is characterized by: In step 43), the method for determining the stability of the quality assessment result is: The method for judging the stability of accuracy quality assessment results is: |Accuracy k+1 ―Accuracy k |≤ε1 Where: ε1 is the maximum deviation allowed for the accuracy quality assessment result; If the difference between the evaluation results of the accuracy of the domain knowledge graph calculated after two consecutive business executions is less than the set maximum deviation ε1, it indicates that the accuracy quality evaluation result is stable; otherwise, it indicates that the accuracy quality evaluation result is unstable; The stability judgment method of completeness quality assessment results is: |Completeness k+1 ―Completeness k |≤ε2 Where: ε2 is the maximum deviation allowed for the completeness quality assessment result; If the difference between the evaluation results of the domain knowledge graph completeness calculated after two consecutive business executions is less than the set maximum deviation ε2, it indicates that the completeness quality evaluation result is stable; otherwise, it indicates that the completeness quality evaluation result is unstable; The method for judging the stability of consistency quality assessment results is: |Consistency k+1 ―Consistency k |≤ε3 Where: ε3 is the maximum deviation allowed for the consistency quality assessment result; If the difference between the evaluation results of the domain knowledge graph consistency calculated after two consecutive business executions is less than the set maximum deviation ε3, it indicates that the consistency quality evaluation result is stable; otherwise, it indicates that the consistency quality evaluation result is unstable; The stability judgment method of robustness quality assessment results is: |Robustness k+1 ―Robustness k |≤ε4 Where: ε4 is the maximum deviation allowed for the robustness quality assessment result; If the difference between the robustness evaluation results of the domain knowledge graph calculated after two consecutive business executions is less than the set maximum deviation ε4, it indicates that the robustness quality evaluation result is stable; otherwise, it indicates that the robustness quality evaluation result is unstable.

7. The vertical domain knowledge graph quality assessment method based on business model drive according to claim 1 is characterized by: In step 5, the method steps for verifying the reliability of the domain knowledge graph quality assessment results through business simulation are as follows: 51) Receive the specific task processed by this business model, and retrieve the domain knowledge related to the business task from the domain knowledge graph; 52) Substitute the retrieved domain knowledge into the business process described by the business model, and calculate the support of the domain knowledge graph for each business process. i ; 53) Summarize the calculation results of the support of the domain knowledge graph for the business process, and obtain the support of the domain knowledge graph for the business task; 54) The support degree of the domain knowledge graph for the business task is combined with the quality assessment score DKG of the domain knowledge graph score Make a comparison and judge the reliability of the evaluation results.

8. The vertical domain knowledge graph quality assessment method based on business model drive according to claim 7 is characterized by: In the step 52), the support degree Support i The ratio of the domain knowledge related to the business process retrieved from the domain knowledge graph to the domain knowledge required to execute the business process is expressed as: Support i =Accuracy S_i +Completeness S_i +Consistency S_i Where: Accuracy S_i The accuracy of the retrieved domain knowledge to support the i-th business process; Completeness S_i The completeness of the retrieved domain knowledge to support the i-th business process; Consistency S_i The retrieved domain knowledge supports the consistency of the i-th business process.

9. The vertical domain knowledge graph quality assessment method based on business model drive according to claim 8 is characterized by: In the step 53), the support degree Support is expressed as: Where: N is the number of business processes in the business model in this field.

10. The vertical domain knowledge graph quality assessment method based on business model drive according to claim 9 is characterized by: In step 54), the quality assessment comprehensive score DKG score It is expressed as: DKG score =Accuracy score +Completeness score +Consistency score +Robustness score Where: Accuracy score Indicates the accuracy score of the domain knowledge graph; Completeness score Indicates the completeness score of the domain knowledge graph; Consistency score Represents the consistency score of the domain knowledge graph; Robustness score Represents the robustness score of the domain knowledge graph; The method for judging the reliability of the evaluation results is: setting the reliability threshold ε of the evaluation results, if |DKG score ―Support|≤ε, it means that the evaluation result of the domain knowledge graph is reliable; If|DKG score ―Support|>ε, it means that the evaluation result of the domain knowledge graph is unreliable.

Images