A knowledge graph entity confidence determination method and related products

Abstract

The application discloses a knowledge graph entity confidence determination method and related products. In the scheme, based on the set of knowledge graph entities to be processed, a multi-perspective reviewer is used to score each entity to obtain a corresponding multi-perspective original score set; based on the multi-perspective original score set of each entity, a divergence value is determined; based on the divergence value of each entity, the category of each entity is determined, and a corresponding manual verification operation is performed based on the category of each entity; the entity after manual verification is subjected to evidence back-link consistency verification to obtain a verification result; based on the multi-perspective original score set, the divergence value and the verification result of each entity, the confidence result of each entity is determined. The application reduces the workload of manual review from multiple links while ensuring the quality of entity confidence evaluation through four types of professional perspective reviewers for full-dimensional pre-screening, divergence-oriented allocation of artificial, evidence back-link verification and three-factor confidence calculation, and fully covers professional verification points.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method for determining the confidence level of entities in a knowledge graph and related products. Background Technology

[0002] As enterprises deepen their digital transformation, the scale of knowledge graphs built based on business document extraction continues to expand, with the scale of graph entities generally reaching thousands to tens of thousands. The quality of entities directly affects the reliability of the output results of downstream intelligent retrieval, business reasoning, risk verification, or access control systems.

[0003] Existing methods for assessing the confidence level of entities in knowledge graphs mainly fall into two categories: one relies on manual review of each entity and the assignment of a confidence score; the other uses a single artificial intelligence model to complete entity extraction and confidence assessment. Both methods have certain drawbacks: when the number of entities reaches thousands, verifying each entity individually requires hundreds of people, making the implementation of routine quality control for large-scale knowledge graphs extremely costly; a single artificial intelligence model focuses on only one dimension, making it prone to missing key verification points and significantly reducing the reliability of the confidence assessment results.

[0004] For large-scale knowledge graph entity evaluation scenarios, how to reduce the cost of manual review while ensuring evaluation quality is an urgent technical problem to be solved. Summary of the Invention

[0005] Based on the above problems, this application provides a method for determining the confidence level of entities in knowledge graphs and related products, aiming to reduce the cost of manual review while ensuring the quality of assessment in large-scale knowledge graph entity evaluation scenarios.

[0006] The embodiments of this application disclose the following technical solutions: The first aspect of this application provides a method for determining entity confidence in a knowledge graph, the method comprising: Based on the entity set of the knowledge graph to be processed, a multi-perspective reviewer is used to score each entity in the entity set of the knowledge graph to be processed, so as to obtain the original multi-perspective score set of the corresponding entity; the multi-perspective reviewer includes a product perspective reviewer, a research and development perspective reviewer, a testing perspective reviewer, and a compliance perspective reviewer. Based on the multi-perspective original score set of each entity, the degree of divergence of the corresponding entity is determined; Based on the divergence value of each entity, the category of each entity is determined, and based on the category of each entity, the corresponding manual verification operation is performed; The entities that have been manually verified are subjected to evidence backlink consistency verification to obtain the corresponding verification results. Based on the multi-perspective original score set, dissonance value and verification results of each entity, the confidence result of each entity is determined.

[0007] Optionally, determining the category of each entity based on its divergence value includes: For any entity: If the divergence value of the current entity is less than or equal to the first preset threshold, then the current entity is classified as a cold zone. If the divergence value of the current entity is greater than the first preset threshold and less than the second preset threshold, then the category of the current entity is the temperature zone. If the divergence value of the current entity is greater than or equal to the second preset threshold, then the current entity is classified as a hot zone.

[0008] Optionally, the step of performing corresponding manual verification operations based on the categories of each entity includes: For any entity: If the current entity is classified as a hotspot, then perform a full manual verification operation. If the current entity's category is "temperature zone", then perform a sampling manual verification operation; If the current entity is in a cold zone, no manual verification is required.

[0009] Optionally, the step of performing evidence back-chain consistency verification on the manually verified entities to obtain the corresponding verification results includes: For any entity: Link the current entity declaration back to the original document evidence, and verify the consistency between the declaration and the evidence through semantic matching and scope verification respectively, to obtain the corresponding semantic matching verification results and scope verification results. A consistency score is calculated based on the semantic matching verification result and the scope verification result. If the consistency score is greater than or equal to a first preset score threshold, the verification passes; if the consistency score is less than the first preset score threshold, the verification fails.

[0010] Optionally, determining the confidence score of each entity based on the multi-perspective original score set, divergence value, and verification results includes: The original multi-perspective score set of each entity is weighted and summed according to the preset perspective weights to obtain the role-weighted average score of each entity. Based on the degree of divergence of each entity, the consensus coefficient of each entity is calculated. Based on the verification results of each entity, the evidence strength coefficient of each entity is determined; The confidence score of an entity is obtained by multiplying the weighted average score of its role, the consensus coefficient, and the evidence strength coefficient.

[0011] Optionally, determining the divergence value of the corresponding entity based on the multi-perspective original rating set of each entity includes: Based on the original multi-perspective rating set of each entity, the rating variance or rating range of the corresponding entity is calculated, and the rating variance or rating range is used as the corresponding divergence value.

[0012] A second aspect of this application provides a knowledge graph entity confidence determination apparatus, the apparatus comprising: The multi-perspective original score set determination module is used to score each entity in the knowledge graph entity set to be processed using a multi-perspective reviewer, based on the entity set to be processed, to obtain the corresponding entity's multi-perspective original score set; the multi-perspective reviewer includes a product perspective reviewer, a research and development perspective reviewer, a testing perspective reviewer, and a compliance perspective reviewer. The divergence value determination module is used to determine the divergence value of the corresponding entity based on the multi-perspective original score set of each entity. The differential verification module is used to determine the category of each entity based on the degree of divergence of each entity, and to perform corresponding manual verification operations based on the category of each entity. The evidence backlink consistency verification module is used to perform evidence backlink consistency verification on entities that have been manually verified, and obtain the corresponding verification results. The confidence result determination module is used to determine the confidence result of each entity based on the multi-perspective original score set, dissonance value and verification result of each entity.

[0013] Optionally, the differential verification module is used for: For any entity: If the divergence value of the current entity is less than or equal to the first preset threshold, then the current entity is classified as a cold zone. If the divergence value of the current entity is greater than the first preset threshold and less than the second preset threshold, then the category of the current entity is the temperature zone. If the divergence value of the current entity is greater than or equal to the second preset threshold, then the current entity is classified as a hot zone.

[0014] A third aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the knowledge graph entity confidence determination method provided in any implementation of the first aspect.

[0015] The fourth aspect of this application provides a processor for running a computer program that, when running, executes a knowledge graph entity confidence determination method as provided in any implementation of the first aspect.

[0016] Compared with the prior art, this application has the following beneficial effects: The knowledge graph entity confidence determination method provided in this application embodiment is based on a set of knowledge graph entities to be processed. It uses a multi-perspective reviewer to score each entity in the set, obtaining a multi-perspective original score set for the corresponding entity. The multi-perspective reviewer includes a product perspective reviewer, a research and development perspective reviewer, a testing perspective reviewer, and a compliance perspective reviewer. Based on the multi-perspective original score set of each entity, the divergence value of the corresponding entity is determined. Based on the divergence value of each entity, the category of each entity is determined, and based on the category of each entity, a corresponding manual verification operation is performed. The manually verified entities undergo evidence backlink consistency verification to obtain the corresponding verification result. Based on the multi-perspective original score set, divergence value, and verification result of each entity, the confidence result of each entity is determined.

[0017] The above method sets up four professional reviewers—product, R&D, testing, and compliance—to score entities independently in parallel, comprehensively covering all professional verification points. This proactively detects defects in most dimensions, reducing the workload of manual review from the source. Without compromising the completeness of the assessment coverage, it compresses the burden of manual processing, ensuring no professional dimension omissions in the final entity assessment and maintaining stable assessment quality. The method quantifies the degree of disagreement through the multi-perspective original score set of each entity, classifies entities based on the degree of disagreement, and implements differentiated manual verification strategies. This achieves targeted allocation of human resources, significantly reducing overall manual input. Entities that have completed manual verification undergo a unified evidence back-chain consistency check, providing quantitative support for entity credibility and reducing the need for repeated manual verification of entity authenticity. This unifies assessment standards and further reduces repetitive manual labor. The final confidence level of an entity is obtained by multiplying the role-weighted average score, consensus coefficient, and evidence strength coefficient. Each factor can be traced back to the multi-perspective original scores, disagreement values, and original evidence documents, eliminating the need for full manual review and indirectly reducing the manual consumption of subsequent quality traceability and secondary verification. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating a method for determining entity confidence in a knowledge graph, provided as an embodiment of this application; Figure 2 This is a schematic diagram of a knowledge graph entity confidence determination device provided in an embodiment of this application. Detailed Implementation

[0020] As described earlier, current methods for assessing the confidence level of entities in knowledge graphs mainly fall into two categories: one relies on manual review of each entity and the assignment of a confidence score; the other uses a single artificial intelligence model to complete entity extraction and confidence assessment. Both methods have certain drawbacks: when the number of entities reaches thousands, verifying each entity individually requires hundreds of people, making the implementation of routine quality control for large-scale knowledge graphs extremely costly; a single artificial intelligence model focuses on only one dimension, making it prone to missing key verification points and significantly reducing the reliability of the confidence assessment results.

[0021] In view of the above problems, this application proposes a method and related products for determining the confidence level of entities in a knowledge graph. Based on a set of entities in a knowledge graph to be processed, a multi-perspective reviewer is used to score each entity in the set, obtaining a set of original multi-perspective scores for the corresponding entity. The multi-perspective reviewer includes a product perspective reviewer, a research and development perspective reviewer, a testing perspective reviewer, and a compliance perspective reviewer. Based on the original multi-perspective scores for each entity, the degree of divergence value of the corresponding entity is determined. Based on the degree of divergence value of each entity, the category of each entity is determined, and based on the category of each entity, a corresponding manual verification operation is performed. The manually verified entities are then subjected to evidence backlink consistency verification to obtain the corresponding verification results. Based on the original multi-perspective scores, divergence values, and verification results of each entity, the confidence level of each entity is determined.

[0022] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0023] See Figure 1 This figure is a flowchart of a method for determining entity confidence in a knowledge graph, provided in an embodiment of this application. Figure 1 As shown, the method includes the following steps: S101. Based on the entity set of the knowledge graph to be processed, use a multi-view examiner to score each entity in the entity set of the knowledge graph to be processed, and obtain the original multi-view score set of the corresponding entity.

[0024] The multi-perspective reviewer includes a product perspective reviewer, a research and development perspective reviewer, a testing perspective reviewer, and a compliance perspective reviewer.

[0025] The product perspective reviewer, R&D perspective reviewer, testing perspective reviewer, and compliance perspective reviewer here are four differentiated professional AI (Artificial Intelligence) reviewers. The product perspective reviewer focuses on the integrity of business processes and coverage of user scenarios; the R&D perspective reviewer focuses on interface specifications, transaction boundaries, and error handling; the testing perspective reviewer focuses on abnormal scenarios, boundary conditions, and regression effects; and the compliance perspective reviewer focuses on audit fields, sensitive data processing, and access control.

[0026] Four types of reviewers score each entity independently and in parallel, with each score ranging from 0 to 100. The four scores for each entity are then aggregated to obtain a unique set of original scores from multiple perspectives. By using differentiated scoring from multiple professional roles, the review blind spots of a single AI are filled, and the initial screening of entities is completed from all dimensions, including product, R&D, testing, and compliance, providing multi-dimensional original evidence for subsequent dispute resolution.

[0027] S102. Based on the multi-perspective original score set of each entity, determine the degree of divergence value of the corresponding entity.

[0028] In one feasible implementation: Based on the original multi-perspective rating set of each entity, the rating variance or rating range of the corresponding entity is calculated, and the rating variance or rating range is used as the corresponding divergence value.

[0029] Rating variance represents the dispersion of character ratings; the larger the rating variance, the greater the difference in ratings among different characters. Rating range is obtained by subtracting the lowest score from the highest score for the same entity, which intuitively reflects the difference between the upper and lower limits of the rating.

[0030] S103. Based on the divergence value of each entity, determine the category of each entity, and perform the corresponding manual verification operation based on the category of each entity.

[0031] In one feasible implementation, the category of each entity is determined based on its divergence value, including: For any entity: If the divergence value of the current entity is less than or equal to the first preset threshold, then the current entity is classified as a cold zone. If the divergence value of the current entity is greater than the first preset threshold and less than the second preset threshold, then the category of the current entity is the temperature zone. If the divergence value of the current entity is greater than or equal to the second preset threshold, then the current entity is classified as a hot zone.

[0032] Pre-configure variance or range thresholds. A higher divergence value indicates more severe conflict in the opinions of multiple roles in the assessment, and a higher risk for the entity. Entities in the hot zone have severe conflicts in the scores of multiple roles, and the highest risk; entities in the warm zone have a moderate degree of disagreement; entities in the cold zone have highly consistent scores among all roles, and the entity has high credibility.

[0033] In one feasible implementation, based on the category of each entity, a corresponding manual verification operation is performed, including: For any entity: If the current entity is classified as a hotspot, then perform a full manual verification operation. If the current entity's category is "temperature zone", then perform a sampling manual verification operation; If the current entity is in a cold zone, no manual verification is required.

[0034] Specifically, entities in hot zones undergo full manual verification, generating high-priority manual work orders that force domain experts to review and correct them; entities in warm zones undergo sampling manual verification, with only a portion of entities being reviewed to balance manual costs and quality; entities in cold zones do not require manual verification, automatically skipping the manual step and directly entering the evidence backlink consistency check.

[0035] The divergence-driven targeted manual verification mechanism concentrates scarce human resources only on entities in hot zones, performs sampling manual verification on entities in warm zones, and does not require manual verification for entities in cold zones. This solves the pain points of high cost of full manual review and omission of high-risk entities by random sampling in existing technologies, and achieves precise allocation of human resources, making the confidence assessment of large-scale knowledge graphs of thousands to tens of thousands of entities feasible for engineering implementation.

[0036] S104. Perform evidence backlink consistency verification on the manually verified entities to obtain the corresponding verification results.

[0037] In one feasible implementation: For any entity: Link the current entity declaration back to the original document evidence, and verify the consistency between the declaration and the evidence through semantic matching and scope verification respectively, to obtain the corresponding semantic matching verification results and scope verification results. A consistency score is calculated based on the semantic matching verification result and the scope verification result. If the consistency score is greater than or equal to a first preset score threshold, the verification passes; if the consistency score is less than the first preset score threshold, the verification fails.

[0038] Semantic matching verification is used to compare the semantic similarity between the entity text description and the original document content; scope verification is used to verify whether the entity's business scope, constraints and original document business boundaries match.

[0039] By anchoring entity claims to original business documents, a binding relationship between entities and evidence is established. This changes the traditional assessment model that relies solely on AI scoring without supporting original text, fundamentally preventing AI from generating unfounded entity information and ensuring that confidence assessments are supported by objective original materials, thus improving the credibility of the assessments. Simultaneously, a dual verification process using semantic matching and business scope is employed. This verifies both the similarity of textual descriptions and whether the entity's business boundaries and constraints match the original text. This eliminates the need for manual word-by-word verification of documents to determine authenticity, automating the process and significantly reducing the repetitive workload of manually verifying entity authenticity, thereby lowering labor costs.

[0040] S105. Based on the multi-perspective original score set, divergence value and verification results of each entity, determine the confidence result of each entity.

[0041] In one feasible implementation: The original multi-perspective score set of each entity is weighted and summed according to the preset perspective weights to obtain the role-weighted average score of each entity. Based on the degree of divergence of each entity, the consensus coefficient of each entity is calculated. Based on the verification results of each entity, the evidence strength coefficient of each entity is determined; The confidence score of an entity is obtained by multiplying the weighted average score of its role, the consensus coefficient, and the evidence strength coefficient.

[0042] The role-weighted average score is obtained by summing the original scores from multiple perspectives based on the preset weights of each professional perspective, with the total weight of all weights being 1. For example, the preset perspective weights can be: R&D perspective 0.4, product perspective 0.3, testing perspective 0.15, and compliance perspective 0.15.

[0043] For example, the consensus coefficient of each entity is calculated based on its degree of divergence value according to the following formula: ; in, Representing entities The consensus coefficient is used to quantify the degree of consistency in scoring across multiple roles, including product, R&D, testing, and compliance. Representing entities Multi-perspective scoring variance is used to quantify the degree of disagreement in scoring from different professional review perspectives; This represents the theoretical maximum variance corresponding to the scoring interval, when the score range for each perspective is 0~100 points. It is 2500.

[0044] The smaller the disagreement in ratings across multiple characters, the lower the rating variance. The smaller the coefficient, the closer the consensus coefficient is to 1, indicating a high degree of agreement among different professional perspectives; the greater the disagreement among multiple roles in scoring, the higher the variance. The larger the coefficient, the closer the consensus coefficient is to 0, indicating that the differences in assessments from different perspectives are huge and the credibility is reduced.

[0045] For example, based on the verification results of each entity, the evidence strength coefficient of each entity is determined, including: For any entity: If the verification result of the current entity is that the verification is passed, then it is determined whether the corresponding consistency score is greater than or equal to the second preset score threshold. If the corresponding consistency score is greater than or equal to the second preset score threshold, then the evidence of the current entity is completely matched and the evidence strength coefficient is 1. If the corresponding consistency score is less than the second preset score threshold but greater than the first score threshold, then the evidence of the current entity is generally matched and the evidence strength coefficient is 0.7. The second preset score threshold is greater than the first score threshold. If the verification result of the current entity is verification failure, then the evidence of the current entity is missing, and the evidence strength coefficient is 0.4.

[0046] For example, the confidence score of an entity is obtained by multiplying the role-weighted average score, consensus coefficient, and evidence strength coefficient of each entity according to the following formula: ; in, Representing entities The role-weighted average score reflects the quality of the entity's basic business operations; Representing entities The consensus coefficient; Representing entities The evidence strength coefficient reflects the strength of the original evidence supporting the entity's statement. Indicates confidence level The confidence level results.

[0047] The final confidence level of an entity is no longer a vague single value, but can be broken down into three parts: weighted average score, consensus coefficient, and evidence strength coefficient. Downstream risk control, access control, and other functions can make refined business decisions based on the details of the three factors. At the same time, the entire calculation process, original scores, and evidence links can be traced back.

[0048] Taking the knowledge graph entity confidence assessment of a supply chain finance platform as an example, the method for determining the confidence level of knowledge graph entities described in the above embodiments will be explained.

[0049] The knowledge graph entity set to be processed consists of 5,000 entities. Four types of professional AI reviewers based on large language models are deployed: product perspective reviewer, R&D perspective reviewer, testing perspective reviewer, and compliance perspective reviewer. Each reviewer scores the entities independently, with a single entity score range of 0 to 100.

[0050] After performing steps S101, S102, and S103, the classification results for each entity are shown in Table 1. Table 1

[0051] As shown in Table 1, the total number of entities requiring manual review is approximately 1200, accounting for only 24% of all entities.

[0052] Assuming an average manual review time of 5 minutes per entity, calculate the total working time for both the fully manual review scheme and the scheme provided in this application's embodiment: Total man-hours for manual item-by-item verification: 5000 × 5 ÷ 60 ≈ 417 hours; The total manual verification time for divergence orientation in this embodiment is 1200 × 5 ÷ 60 ≈ 100 hours, plus a small amount of sampling secondary verification time, totaling approximately 110 hours.

[0053] Work time saving percentage: (417-110)÷417≈73%; After manual verification, all entities enter the evidence backlink verification process: the entity declaration is accurately anchored to the original business document path and corresponding line number, semantic matching verification and business scope verification are performed respectively, and the consistency score is calculated comprehensively.

[0054] Taking entity E007 as an example, the original scores of entity E007 from each perspective of the reviewer are as follows: product perspective 95 points, R&D perspective 100 points, testing perspective 90 points, and compliance perspective 85 points. The preset perspective weights are: product perspective weight 0.30, R&D perspective weight 0.40, testing perspective weight 0.15, and compliance perspective weight 0.15. The calculated role-weighted average score is 95.

[0055] The variance of entity E007's rating is 25, and the theoretical maximum variance corresponding to the rating range of 0~100 is 2500. The calculated consensus coefficient is 0.99.

[0056] Entity E007 has a consistency score of 92, which is greater than the second preset score threshold of 80. The evidence is completely matched, and the corresponding evidence strength coefficient is 1.0.

[0057] Multiplying the three factors of entity E007's role weighted average score, consensus coefficient, and evidence strength coefficient together, the confidence score of entity E007 is 94.

[0058] The knowledge graph entity confidence determination method provided in this application embodiment is based on a set of knowledge graph entities to be processed. It uses a multi-perspective reviewer to score each entity in the set, obtaining a multi-perspective original score set for the corresponding entity. The multi-perspective reviewer includes a product perspective reviewer, a research and development perspective reviewer, a testing perspective reviewer, and a compliance perspective reviewer. Based on the multi-perspective original score set of each entity, the divergence value of the corresponding entity is determined. Based on the divergence value of each entity, the category of each entity is determined, and based on the category of each entity, a corresponding manual verification operation is performed. The manually verified entities undergo evidence backlink consistency verification to obtain the corresponding verification result. Based on the multi-perspective original score set, divergence value, and verification result of each entity, the confidence result of each entity is determined.

[0059] The above method sets up four professional reviewers—product, R&D, testing, and compliance—to score entities independently in parallel, comprehensively covering all professional verification points. This proactively detects defects in most dimensions, reducing the workload of manual review from the source. Without compromising the completeness of the assessment coverage, it compresses the burden of manual processing, ensuring no professional dimension omissions in the final entity assessment and maintaining stable assessment quality. The method quantifies the degree of disagreement through the multi-perspective original score set of each entity, classifies entities based on the degree of disagreement, and implements differentiated manual verification strategies. This achieves targeted allocation of human resources, significantly reducing overall manual input. Entities that have completed manual verification undergo a unified evidence back-chain consistency check, providing quantitative support for entity credibility and reducing the need for repeated manual verification of entity authenticity. This unifies assessment standards and further reduces repetitive manual labor. The final confidence level of an entity is obtained by multiplying the role-weighted average score, consensus coefficient, and evidence strength coefficient. Each factor can be traced back to the multi-perspective original scores, disagreement values, and original evidence documents, eliminating the need for full manual review and indirectly reducing the manual consumption of subsequent quality traceability and secondary verification.

[0060] Based on the knowledge graph entity confidence determination method described in the preceding embodiments, this application also provides a knowledge graph entity confidence determination device. Figure 2 This is a schematic diagram of the device. Figure 2 As shown, the knowledge graph entity confidence determination device includes: The multi-perspective original score set determination module 201 is used to score each entity in the knowledge graph entity set to be processed using a multi-perspective reviewer, based on the entity set to be processed, to obtain the multi-perspective original score set of the corresponding entity; the multi-perspective reviewer includes a product perspective reviewer, a research and development perspective reviewer, a testing perspective reviewer, and a compliance perspective reviewer. The divergence value determination module 202 is used to determine the divergence value of the corresponding entity based on the multi-perspective original score set of each entity. The differential verification module 203 is used to determine the category of each entity based on the degree of divergence of each entity, and to perform the corresponding manual verification operation based on the category of each entity. The evidence backlink consistency verification module 204 is used to perform evidence backlink consistency verification on entities that have been manually verified, and obtain the corresponding verification results. The confidence result determination module 205 is used to determine the confidence result of each entity based on the multi-perspective original score set, the degree of divergence value and the verification result of each entity.

[0061] Optionally, the differential verification module is used for: For any entity: If the divergence value of the current entity is less than or equal to the first preset threshold, then the current entity is classified as a cold zone. If the divergence value of the current entity is greater than the first preset threshold and less than the second preset threshold, then the category of the current entity is the temperature zone. If the divergence value of the current entity is greater than or equal to the second preset threshold, then the current entity is classified as a hot zone.

[0062] Optionally, the differential verification module is used for: For any entity: If the current entity is classified as a hotspot, then perform a full manual verification operation. If the current entity's category is "temperature zone", then perform a sampling manual verification operation; If the current entity is in a cold zone, no manual verification is required.

[0063] Optionally, the evidence back-chain consistency verification module is used for: For any entity: Link the current entity declaration back to the original document evidence, and verify the consistency between the declaration and the evidence through semantic matching and scope verification respectively, to obtain the corresponding semantic matching verification results and scope verification results. A consistency score is calculated based on the semantic matching verification result and the scope verification result. If the consistency score is greater than or equal to a first preset score threshold, the verification passes; if the consistency score is less than the first preset score threshold, the verification fails.

[0064] Optionally, the confidence result determination module is used for: The original multi-perspective score set of each entity is weighted and summed according to the preset perspective weights to obtain the role-weighted average score of each entity. Based on the degree of divergence of each entity, the consensus coefficient of each entity is calculated. Based on the verification results of each entity, the evidence strength coefficient of each entity is determined; The confidence score of an entity is obtained by multiplying the weighted average score of its role, the consensus coefficient, and the evidence strength coefficient.

[0065] Optionally, the divergence value determination module is used to: Based on the original multi-perspective rating set of each entity, the rating variance or rating range of the corresponding entity is calculated, and the rating variance or rating range is used as the corresponding divergence value.

[0066] Furthermore, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the knowledge graph entity confidence determination method as described in any of the method embodiments.

[0067] Furthermore, this application embodiment also provides a processor for running a computer program, which executes the knowledge graph entity confidence determination method as described in any of the foregoing method embodiments.

[0068] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separate. The components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment solution according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0069] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for determining entity confidence in a knowledge graph, characterized in that, include: Based on the entity set of the knowledge graph to be processed, a multi-perspective reviewer is used to score each entity in the entity set of the knowledge graph to be processed, so as to obtain the original multi-perspective score set of the corresponding entity; the multi-perspective reviewer includes a product perspective reviewer, a research and development perspective reviewer, a testing perspective reviewer, and a compliance perspective reviewer. Based on the multi-perspective original score set of each entity, the degree of divergence of the corresponding entity is determined; Based on the divergence value of each entity, the category of each entity is determined, and based on the category of each entity, the corresponding manual verification operation is performed; The entities that have been manually verified are subjected to evidence backlink consistency verification to obtain the corresponding verification results. Based on the multi-perspective original score set, dissonance value and verification results of each entity, the confidence result of each entity is determined.

2. The method according to claim 1, characterized in that, The process of determining the category of each entity based on its divergence value includes: For any entity: If the divergence value of the current entity is less than or equal to the first preset threshold, then the current entity is classified as a cold zone. If the divergence value of the current entity is greater than the first preset threshold and less than the second preset threshold, then the category of the current entity is the temperature zone. If the divergence value of the current entity is greater than or equal to the second preset threshold, then the current entity is classified as a hot zone.

3. The method according to claim 2, characterized in that, The step of performing corresponding manual verification operations based on the categories of each entity includes: For any entity: If the current entity is classified as a hotspot, then perform a full manual verification operation. If the current entity's category is "temperature zone", then perform a sampling manual verification operation; If the current entity is in a cold zone, no manual verification is required.

4. The method according to claim 1, characterized in that, The step of performing evidence backlink consistency verification on the manually verified entities to obtain the corresponding verification results includes: For any entity: Link the current entity declaration back to the original document evidence, and verify the consistency between the declaration and the evidence through semantic matching and scope verification respectively, to obtain the corresponding semantic matching verification results and scope verification results. A consistency score is calculated based on the semantic matching verification result and the scope verification result. If the consistency score is greater than or equal to a first preset score threshold, the verification passes; if the consistency score is less than the first preset score threshold, the verification fails.

5. The method according to claim 1, characterized in that, The confidence score of each entity is determined based on the multi-perspective original score set, divergence value, and verification results, including: The original multi-perspective score set of each entity is weighted and summed according to the preset perspective weights to obtain the role-weighted average score of each entity. Calculate the consensus coefficient of each entity based on its degree of divergence. Based on the verification results of each entity, the evidence strength coefficient of each entity is determined; The confidence score of an entity is obtained by multiplying the weighted average score of its role, the consensus coefficient, and the evidence strength coefficient.

6. The method according to claim 1, characterized in that, The determination of the divergence value of the corresponding entity based on the multi-perspective original score set of each entity includes: Based on the original multi-perspective rating set of each entity, the rating variance or rating range of the corresponding entity is calculated, and the rating variance or rating range is used as the corresponding divergence value.

7. A device for determining the confidence level of entities in a knowledge graph, characterized in that, include: The multi-perspective original score set determination module is used to score each entity in the knowledge graph entity set to be processed using a multi-perspective reviewer, based on the entity set to be processed, to obtain the corresponding entity's multi-perspective original score set; the multi-perspective reviewer includes a product perspective reviewer, a research and development perspective reviewer, a testing perspective reviewer, and a compliance perspective reviewer. The divergence value determination module is used to determine the divergence value of the corresponding entity based on the multi-perspective original score set of each entity. The differential verification module is used to determine the category of each entity based on the degree of divergence of each entity, and to perform corresponding manual verification operations based on the category of each entity. The evidence backlink consistency verification module is used to perform evidence backlink consistency verification on entities that have been manually verified, and obtain the corresponding verification results. The confidence result determination module is used to determine the confidence result of each entity based on the multi-perspective original score set, dissonance value and verification result of each entity.

8. The apparatus according to claim 7, characterized in that, The differential verification module is used for: For any entity: If the divergence value of the current entity is less than or equal to the first preset threshold, then the current entity is classified as a cold zone. If the divergence value of the current entity is greater than the first preset threshold and less than the second preset threshold, then the category of the current entity is the temperature zone. If the divergence value of the current entity is greater than or equal to the second preset threshold, then the current entity is classified as a hot zone.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the knowledge graph entity confidence determination method as described in any one of claims 1-6.

10. A processor, characterized in that, Used to run a computer program, which, when running, executes the knowledge graph entity confidence determination method as described in any one of claims 1-6.

Images