Two-way dynamic fact anchoring verification method and system for content generated by large language model

Abstract

The application discloses a kind of big language model generation content's two-way dynamic fact anchoring verification method and system, belong to artificial intelligence and natural language processing technical field.For part fact verification scheme can only verify " right or wrong " of generated content, it is difficult to verify the problem of " full or not " of key fact, the two-way closed loop of positive verification and reverse verification is established: positive verification is compared with external knowledge base entity for semantic consistency of atomic declaration, and numerical value type declaration is forced to determine according to numerical value difference rate;Reverse verification extracts the list of key facts related to user questions from the knowledge base, checks if it is directly mentioned or implicitly covered by the answer, and generates a missing list;Based on the positive pass rate and reverse coverage rate, the comprehensive confidence is calculated, and the graded reference is output combined with the domain adaptive threshold.The application outputs fact verification results, evidence pointers and graded references, does not cache retry session, and does not perform final shunt based on model output layer uncertainty, and the final disposal can be covered by downstream cognitive honesty decision module.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and natural language processing technology, specifically relating to a method and system for bidirectional dynamic fact anchoring verification of content generated by a large language model. More particularly, it involves a technical solution that achieves collaborative verification of factual accuracy and answer completeness through forward semantic consistency comparison, numerical declaration difference rate verification, reverse key fact coverage check, comprehensive confidence calculation, and domain-adaptive threshold adjustment. This invention outputs verification results and a hierarchical reference, does not perform final OUTPUT / CAUTION / ABSTAIN routing, and does not handle session caching for retrying after a rejected answer. Background Technology

[0002] As the application of large language models expands into high-risk fields such as healthcare, law, and finance, the factual accuracy of the content generated by these models has become a core bottleneck affecting their secure implementation. Some search enhancement generation or fact verification schemes employ a one-way verification mechanism: comparing the model-generated answers with an external knowledge base to verify the accuracy of the facts presented in the answers. While this type of mechanism can detect some errors, it has the following limitations.

[0003] Limitation 1: It can only verify "correctness," not "completeness." One-way verification mainly checks whether the content already appearing in the answer is consistent with the knowledge base. For fidelity biases such as model avoidance of key questions, selective omissions, and topic shifting, if the corresponding facts are not present in the answer, one-way comparison lacks an entry point for inspection. For example, if a user asks, "What are the early symptoms of diabetes? How to prevent it?", and the model only answers symptoms while omitting the prevention part, one-way verification usually cannot identify this omission.

[0004] Deficiency 2: Lack of quantifiable assessment of completeness. Some solutions fail to establish a reverse coverage check mechanism from the knowledge base to the generated content, making it impossible to express "whether the key facts related to user issues are adequately covered" as a calculable indicator, resulting in completeness judgment remaining at the qualitative experience level.

[0005] Defect 3: The detection dimension is too singular, easily confusing factual errors with fidelity bias. Factual errors occur when the answer itself contradicts the knowledge base; fidelity bias occurs when a single sentence in the answer may be correct, but the overall answer fails to cover the key facts of the question. The failure mechanisms of the two are different. If the same indicator, threshold, and intervention method are used, it is easy to result in insufficient interception of high-risk factual errors and excessive interception of low-risk omissions.

[0006] Defect 4: The verification standard lacks domain adaptability. The requirements for the accuracy and completeness of facts vary significantly across scenarios such as casual conversation, general knowledge, professional academics, and medical, legal, and financial fields; a fixed threshold may lead to excessive interception in lenient scenarios and missed detections in high-risk scenarios.

[0007] Therefore, in the aspect of fact verification for LLM-generated content in the prior art, there is an urgent need for a two-way closed-loop verification mechanism that simultaneously covers forward consistency comparison and reverse key fact coverage check. This mechanism shall interface with atomic statement extraction, numerical statement verification, domain threshold adjustment and subsequent hierarchical decision-making; the present invention does not replace external knowledge base construction, retrieval ranking models or underlying language model training. Summary of the Invention

[0008] The problems to be solved by the present invention are: how to simultaneously verify whether the generated facts are "correct" and whether the key facts are "complete"; how to quantify answer completeness; how to distinguish factual errors from faithfulness deviations; how to perform mandatory determination on numerical statements; and how to dynamically adjust verification thresholds according to domains.

[0009] S0: Domain adaptive threshold adjustment. A lightweight domain classifier is used to identify the domain to which a question belongs, and set forward verification thresholds T1 / T2, hierarchical reference thresholds T_pass / T_warn, and numerical difference rate threshold T_num; the thresholds are configurable parameters, and preferred examples are given in the embodiments.

[0010] S1: Forward verification. Receive the atomic statement set C={c1,...,cn}, perform semantic consistency comparison of each atomic statement ci with the corresponding entity E in the external knowledge base, and calculate Sim(ci,E)=cos(vi,ve). When Sim≥T1, it is determined as verified; when T2≤Sim<T1, it is determined as questionable; when Sim<T2, it is determined as contradictory. For numerical statements, the measurement units and comparison calibers are unified first, and when N_kb≠0, the difference rate is calculated as |N_claim−N_kb| / N_kb×100%; when the difference rate exceeds T_num, it is directly determined as contradictory; when N_kb=0, the dimensions are inconsistent, or the interval calibers cannot be unified, no separate mandatory numerical contradiction is triggered, and processing is performed only according to semantic rules.

[0011] S2: Reverse verification. Extract a key fact list K={k1,...,km} related to the user question Q from the external knowledge base; check one by one whether each key fact kj is directly mentioned or implicitly covered in the model answer A. Direct mention includes string matching or semantic equivalence; implicit coverage includes the answer containing the subordinate concept, synonymous expression, instantiated expression of kj, or being inferentially covered via the subClassOf / type path of the knowledge graph. Uncovered facts are added to the omission list M and sorted by core attributes / secondary attributes.

[0012] S3: Two-way comprehensive scoring. Calculate the forward verification pass rate F = number of verified atomic statements / total number of atomic statements; when the total number of atomic statements is 0, F=0. Calculate the reverse coverage rate V = number of covered key facts / total number of key facts; when the total number of key facts is 0, V=1. The comprehensive confidence S_conf = αF+βV, where α+β=1, 0<α<1, 0<β<1. Output the graded reference according to S_conf and the domain threshold: S_conf≥T_pass is reference pass; T_warn≤S_conf<T_pass is reference warning with an omission list attached; S_conf<T_warn is reference rejection. The graded reference is the reference opinion of the fact verification layer, which can be overwritten by the downstream cognitive honesty decision module based on factors such as model internal uncertainty.

[0013] The present invention further provides a two-way dynamic fact anchoring and verification system, comprising a domain adaptation module, a forward verification module, a reverse verification module, and a two-way comprehensive scoring module; the output objects at least include the verification status of each statement, evidence pointer, key fact list, omission list, comprehensive confidence, threshold version, and graded reference.

[0014] 3.3 Data objects and interface fields The ClaimVerdict field of the forward verification output object includes: claim_id, status∈{verified,suspected,contradicted}, sim_score, numeric_diff_rate, unit_normalized, kb_entity_id, evidence_pointer, threshold_version, kb_version. The CoverageResult field of the reverse verification output object includes: key_fact_id, fact_text, priority∈{high,low}, covered∈{direct,implicit,false}, cover_path, missing_list_id, evidence_pointer. The VerificationResult field of the comprehensive output includes: F, V, S_conf, alpha, beta, domain, threshold_version, graded_reference∈{pass_reference,warn_reference,reject_reference}, report_ref, trace_id. All objects carry kb_version, model_version and threshold_version to support version conflict backtracking.

[0015] The standardized Verification API accepts a Q, a ClaimStream or an array of claims, a domain label, a threshold version, and a trace_id, and returns a VerificationResult. The interface supports idempotent keys, timeouts, rate limiting, error codes, and trace_id. Error codes must include at least KB_ENTITY_NOT_FOUND, NUMERIC_UNIT_MISMATCH, GRAPH_PATH_CYCLE, THRESHOLD_VERSION_MISSING, RATE_LIMITED, and EVIDENCE_POINTER_INVALID.

[0016] 3.4 Rules for Handling Numeric Declarations Numerical declaration extraction employs unit-aware extraction: first, the numerical value, unit, comparison operator, and interval are identified, then converted to a unified unit. If the declaration is an interval, the criteria must be pre-defined: the upper limit, lower limit, or median must be used for comparison; if the standard value of the knowledge base is an interval, it is calculated based on the overlap rate or boundary difference, and cannot be simply replaced by a single-point difference. If N_kb=0, a contradiction is only judged when the semantic similarity is below the contradiction threshold, and no strong numerical judgment is triggered; if the unit cannot be converted, NUMERIC_UNIT_MISMATCH is recorded and processed according to semantic rules. When the difference rate exceeds T_num, even if the semantic similarity is high, it is directly judged as contradicted, and the numerical evidence is written to evidence_pointer.

[0017] 3.5 Reverse Coverage and Implicit Coverage Rules Direct mention includes: key facts matching the answer fragment string; key facts and answer fragments being semantically equivalent and having the same entity set. Implicit coverage includes two auditable paths: first, a semantic path, where the semantic similarity between the key facts and the answer is not lower than the configurable threshold θ_impl, and the answer contains subordinate concepts, synonyms, or instantiations of the key facts; second, a graph path, where the external knowledge graph has a subClassOf or type inference path from the concept in the answer to the key facts, and the path is acyclic and its length does not exceed a preset upper limit. If both paths are unavailable or not hit, it is determined that there is no coverage and it is added to the omission list M; higher priority facts are ranked before lower priority facts.

[0018] 3.6 Boundary Conditions, Degradation, and Anomalies When the total number of atomic declarations is 0, F=0 and forward verification does not perform any comparisons except for empty set checks; when the total number of key facts is 0, V=1, indicating no missing requirements. When T_warn=T_pass, the warning interval is empty, and only reference passed or reference rejected is output. If the semantic vector service is unavailable, the semantic path in the implicit coverage is downgraded to string and entity matching; if the knowledge graph service is unavailable, the graph path is skipped and graph_fallback=true is marked; if the knowledge base entity is not found, the declaration status is suspected or contradicted, determined by the domain policy, and KB_ENTITY_NOT_FOUND is recorded. All downgrades must not delete evidence pointers.

[0019] (1) Two-way closed loop: Simultaneously verify "correctness" and "completeness". (2) Quantifiable completeness: Reverse coverage makes omissions computable and sortable. (3) Strong numerical judgment: Contradictions can be directly determined for numerical claims of the same unit and N_kb≠0. (4) Domain adaptive: Thresholds are configured according to scenarios. (5) Version auditable: ClaimVerdict, CoverageResult, and VerificationResult all have threshold versions and knowledge base versions. (6) Clear boundaries: Output verification results and hierarchical references, without competing for boundaries with downstream final routing, session retry, and scheduling learning. Attached Figure Description

[0020] Figure 1 The flowchart of the two-way dynamic fact anchoring verification method shows the S0 domain adaptive threshold adjustment, S1 forward verification, S2 reverse verification, S3 two-way comprehensive scoring and graded reference output.

[0021] Figure 2 This is a schematic diagram of a two-way closed loop for forward and reverse verification, showing the relationship between atomic declarations, external knowledge base, forward verification output, reverse verification output, and overall confidence level.

[0022] Figure 3 The logic diagram for hierarchical reference and domain-adaptive threshold adjustment shows examples of thresholds for three categories: casual conversation / common sense, professional / academic, and medical / legal / financial, along with hierarchical references for pass / warning / refusal.

[0023] Figure 4 The system module structure diagram shows the domain adaptive module 404, the forward verification module 401, the reverse verification module 402, the bidirectional comprehensive scoring module 403, and the system boundary. Detailed Implementation

[0024] Example 1: Medical Value Contradiction. A user queries "What is the maximum / minimum mmol / L required for fasting blood glucose in the diagnostic criteria for diabetes?", identified as medical / high-risk. A strict threshold example is used: T1=0.95, T2=0.80, T_pass=T_warn=0.95, T_num=1%. The LLM (Local Level Module) answers "Fasting blood glucose ≥ 6.0 mmol / L", while the knowledge base standard value is "≥ 7.0 mmol / L". After unifying units, the difference rate = |6.0−7.0| / 7.0 = 14.3% > 1%, directly indicating a contradiction. In reverse validation, the key facts k1 (fasting blood glucose threshold) are covered, and k2 (2-hour postprandial blood glucose threshold) are omitted. A list of omissions is output with a low overall confidence level. Because T_pass=T_warn, the system only outputs "Reference Pass / Reference Reject" for downstream final decision-making.

[0025] Example 2: Integrity Warning. A user queries "Explain quantum entanglement and describe its applications and commercial prospects." The forward validation has passed, but the reverse critical fact list includes three items: conceptual explanation, application examples, and commercial prospects. The answer only covers the first item. With F=1, V=1 / 3, α=0.6, β=0.4, then S_conf=0.6×1+0.4×1 / 3=0.733; professional / academic standard threshold example T_pass=0.90, T_warn=0.60. A reference warning is output along with an omission list. If downstream users hesitate and ultimately change to ABSTAIN based on the model, this invention retains the evidence pointers and validation results for auditing.

[0026] Example 3: Implicit Coverage. Key fact kj = "Capital of China", answer A = "Beijing". Direct mention is not valid; if the semantic vector similarity is higher than the configurable threshold θ_impl and the answer contains instance representations, or if the graph contains instance relationship paths, then implicit coverage is determined and it is not included in the omission list. If the graph path contains a cycle or exceeds the length limit, then GRAPH_PATH_CYCLE is recorded and it is treated as not covered.

[0027] Example 4: Zero Denominator and Empty List. If the upstream does not generate any atomic claims due to the lack of entities, then F=0, and forward_status=empty_claims in VerificationResult; if the knowledge base does not extract key facts, then V=1, and reverse_status=no_key_facts. The system must not interpret F=0 as all passes, nor V=1 as high coverage quality.

[0028] Example 5: API Sample. Request: POST / verify, body={query:"...", claims:[...],domain_tag:"medical", threshold_version:"th-med-2026q1", idempotency_key:"idem-1", trace_id:"tr-9"}. Response: {F:0.0,V:0.333,S_conf:0.1332,graded_reference:"reject_reference", claim_verdicts:[...], coverage:{covered:1,missing:2,missing_list:[...]}, threshold_version:"th-med-2026q1", kb_version:"kb-2026-08", report_ref:"rep-123"}.

[0029] Example 6: Simulation Comparison (Simulation settings, not actual testing). 500 queries were constructed, 200 of which contained fidelity omissions. The control group only performed forward validation, with omission detection depending on whether related erroneous statements appeared in the answers; this invention adds a reverse coverage check. Under the same knowledge base and retrieval conditions, simulations show that this invention significantly improves recall for selective omissions compared to the forward-only approach, and the increased latency mainly comes from the construction of the key fact list; caching one-hop / two-hop query templates can further reduce latency. This simulation description should be replaced with real logs or controlled experiments before formal submission.

Claims

1. A bidirectional dynamic fact-anchoring verification method for content generated by a large language model, characterized in that, comprising: Domain-adaptive threshold adjustment: identifying the domain to which a problem belongs, and setting a positive verification threshold, a hierarchical reference threshold, and a numerical difference rate threshold; Positive verification: receiving a set of atomic statements, performing semantic consistency comparison between each atomic statement and the corresponding entity in the external knowledge base, and outputting a classification result of verified, questionable or contradictory according to semantic similarity; For numerical statements, unifying the measurement unit and calculating the difference rate between the statement value and the standard value in the knowledge base when the standard value in the knowledge base is not zero, and directly determining the result as contradictory when the difference rate exceeds the preset numerical difference rate threshold; Reverse verification: extracting a list of key facts related to the user's question from the external knowledge base, checking one by one whether each key fact is directly mentioned or implicitly covered in the model answer, adding the uncovered key facts to an omission list and sorting them by importance; Two-way comprehensive scoring: calculating the positive verification pass rate and the reverse coverage rate, where the comprehensive confidence = α × positive verification pass rate + β × reverse coverage rate, α+β=1, 0<α<1, and 0<β<1; when the total number of atomic statements is zero, the positive verification pass rate is set to zero, and when the total number of key facts is zero, the reverse coverage rate is set to one; outputting a hierarchical reference according to the comprehensive confidence and the domain-adaptive threshold.

2. The method according to claim 1, characterized in that, The semantic consistency comparison adopts the cosine similarity of semantic vectors: Sim(ci,E)=cos(vi,ve), where vi is the semantic vector of the atomic statement, and ve is the semantic vector of the entity in the knowledge base.

3. The method according to claim 1, characterized in that, The positive verification classification is: verified when Sim≥the first threshold T1; questionable when the second threshold T2≤Sim<T1; contradictory when Sim<T2; preferably T1=0.85, T2=0.

60.

4. The method according to claim 1, characterized in that, The numerical difference rate threshold is 5% in the science / engineering domain, and 1% in the finance / law / medical domain.

5. The method according to claim 1, characterized in that, The determination of implicit coverage includes: the semantic similarity between the key fact and the answer is not lower than the configurable implicit coverage threshold θ_impl, and the answer contains a subordinate concept, synonymous expression or instantiated expression of the key fact; or there exists a subClassOf / type inference path from the concept in the answer to the key fact through the knowledge graph.

6. The method according to claim 1, characterized in that, The key fact list includes core attribute facts that must be covered to answer the user's question and supplementary secondary attribute facts, where the core attribute facts are set to high priority, and the secondary attribute facts are set to low priority.

7. The method according to claim 1, characterized in that, Preferably α=0.6, β=0.4; the hierarchical reference includes reference pass, reference warning and reference rejection; when T_warn=T_pass, the warning interval is empty, and only reference pass or reference rejection is output.

8. The method according to claim 1, characterized in that, The hierarchical reference is a reference opinion of the fact verification layer, and does not exclude the downstream cognitive honesty decision module from overriding the final disposition in combination with the uncertainty of the model output layer; the method does not cache rejection / warning retry sessions.

9. The method according to claim 1, characterized in that, The output object carries a threshold version, a knowledge base version, a model version and an evidence pointer; when the semantic vector service or the knowledge graph service is unavailable, the service is degraded to string / entity matching or graph path skipping respectively, and a degradation flag is retained.

10. A bidirectional dynamic fact-anchoring verification system for content generated by a large language model, characterized in that, The system includes a domain adaptation module, a forward verification module, a reverse verification module, and a two-way comprehensive scoring module; the system executes the method described in any one of claims 1–9 and outputs verification status, evidence pointers, a list of key facts, a list of omissions, comprehensive confidence level, and graded reference.

Images