Illusion elimination method for legal domain large language model based on uncertainty theory

Abstract

The application is suitable for the field of artificial intelligence and legal information technology, and provides a legal field large language model illusion elimination method based on uncertainty theory, which comprises the following steps: legal task object receiving and preprocessing; large language model initial reasoning and answer generation; legal logic compilation; multi-channel form verification; uncertainty theory reliability calibration and fusion: based on the uncertainty measure product axiom, the reliability of each channel is fused into calibrated reliability; risk assessment and illusion judgment output; the application combines uncertainty theory with formal logic verification, can effectively detect and eliminate large language model illusions in the processing of legal provisions, evidence texts and legal affairs related fact event statements, and significantly improves the accuracy and reliability of the processing results of legal provisions, evidence texts and legal affairs related fact event statements.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and legal information technology, specifically to a method for eliminating illusions in the legal field based on the large language model of uncertainty theory. Background Technology

[0002] With the rapid development of large language models in the field of natural language processing, they have been widely applied in legal intelligence scenarios such as legal question answering, contract review, case analysis, and legal provision retrieval. Leveraging their pre-training capabilities on massive legal corpora, large language models can generate logically fluent and formatted legal analysis texts. However, large language models suffer from an inherent "illusion" problem, generating content that appears reasonable but is actually factually or logically incorrect. In the legal field, this illusion manifests as fabricating non-existent legal provisions, incorrectly citing legal provision numbers, confusing the scope of application of legal concepts, logical jumps or contradictions in reasoning, and exhibiting high credibility in incorrect answers.

[0003] According to the AIIA's "Framework for Evaluating Illusions in Generative Artificial Intelligence Model Applications" (standard number AIIA / T0294-2026), an illusion refers to "content generated by a generative artificial intelligence model that appears reasonable but violates objective facts, input context, or logical rules, and may lead to cognitive misunderstandings or decision-making risks." The illusion of large language models in the processing of legal clauses, evidentiary texts, and statements of facts and events related to legal affairs is particularly harmful in four typical business scenarios: precise matching of legal provisions, judgment of legal version evolution, extraction of key facts from judgments, and extraction of substantive information from cases. It may directly lead to erroneous judgments.

[0004] Existing methods for mitigating illusions in large language models mainly include: self-consistency methods, which select high-frequency answers through multiple sampling and voting, but cannot distinguish situations where "the majority of people make the same mistake," and in complex scenarios such as legal multiple-choice questions, the voting mechanism may amplify systematic bias; validator methods, which use a second large language model to review the initial answers, but the validator model itself may still be affected by homology bias; and retrieval-enhanced generation methods, which, although they can introduce external knowledge, lack logical consistency checks on the reasoning process itself, and cannot detect illusions when the model correctly retrieves legal provisions but applies them incorrectly. All of these methods share the following limitations: they lack formal verification capabilities for the legal processing chain, cannot quantify the credibility of each step in the reasoning process, do not incorporate the verification of the authenticity of legal citations into the illusion detection process, and are unable to effectively distinguish between illusions of "incorrect answers" and "unreliable reasoning processes," severely restricting the credible application of large language models in the legal field. Therefore, in response to the above situation, there is an urgent need to provide a method for eliminating illusions in large language models in the legal field based on uncertainty theory to overcome the shortcomings in current practical applications. Summary of the Invention

[0005] The purpose of this invention is to provide a method for eliminating illusions in the legal field based on the large language model of uncertainty theory, which effectively solves the problems mentioned in the background art.

[0006] This invention is implemented as follows: a method for eliminating illusions in the legal field based on a large language model of uncertainty theory, the method comprising the following steps: Legal task object reception and preprocessing: Receive legal clauses, evidence texts, or statements of legal-related facts and events as task objects; parse the text content, candidate conclusions, and verification instructions; construct a structured representation of the legal task object; and retrieve relevant legal provisions from the legal knowledge base as contextual enhancement for the search. Initial reasoning and answer generation of the large language model: Structured legal questions and enhanced retrieval context are injected into the prompt template, and the large language model is invoked to generate an initial answer that includes option selection, reasoning process, model self-assessment confidence, and cited legal provisions; Legal logic compilation: The legal logic compiler, assisted by a large language model, is invoked to compile the natural language reasoning of the initial answer into an intermediate legal logic representation. The intermediate representation includes a set of atomic legal propositions, a set of legal rules, a sequence of reasoning chain steps, a list of cited legal provisions, and an option-proposition mapping relationship. Multi-channel formal verification: The intermediate representation of the legal logic is verified through at least two of the following three independent channels, and the at least two channels must include a response set procedural symbolic reasoning consistency verification channel: The first channel is for verifying legal provisions from the legal knowledge base; The second channel is for verifying the consistency of symbolic reasoning in the answer set procedure. The third channel is truth propagation verification based on the inference chain of uncertainty theory; Uncertainty theory reliability calibration and fusion: The reliability of each verification channel is regarded as the uncertainty measure of independent uncertain events, and the multi-channel uncertainty theory reliability is fused into calibration reliability based on the uncertainty measure product axiom; Risk assessment and hallucination determination output: Based on the calibration reliability, the consistency results of the answer set procedure and the reasonableness of the inference chain assignment, a comprehensive risk score is calculated. When the comprehensive risk score exceeds the preset threshold, it is determined that there is hallucination and the answer abandonment mechanism is triggered; otherwise, the verified final answer is output.

[0007] As a further aspect of the present invention: the steps of receiving and preprocessing the legal task object specifically include: The legal task object is structured and analyzed to extract the unique task identifier, the text to be verified, the candidate conclusions, the legal jurisdiction, the legal field classification, the time sensitivity indicator and the list of legal provisions, and to construct the legal task data items; Based on the question stem and option text, the legal citation regular pattern matching is used to identify the legal names and article numbers involved; The full text of the matching legal provisions is retrieved sequentially through the cascading architecture of the structured legal database adapter and the local rule base adapter, and used as the context for enhancing the retrieval to inject subsequent prompt templates.

[0008] As a further aspect of the present invention: the legal logic compilation step specifically includes: Construct legal formal logic compilation prompts and inject the question stems, options, and initial answers of the large language model into the compilation prompt template; The compiler prompts are executed by calling the large language model for verification, and a JSON output containing a list of propositions, a list of rules, a chain of inferences, cited legal provisions, and option-proposition mappings is obtained. The compilation output is parsed and semantically verified, including: checking whether the premise identifier of each step in the reasoning chain exists in the proposition set or rule set, checking whether the conclusion identifier exists in the proposition set, correcting the truth value range to the [0,1] interval, and recording all compilation errors.

[0009] As a further aspect of the present invention: the atomic legal proposition includes a unique proposition identifier, the proposition's natural language text, a predicate, a subject, a truth value, a source, and a negation flag; The legal rules include a unique rule identifier, the name of the legal basis, the rule text, a list of antecedent proposition identifiers, a consequent proposition identifier, a list of exceptions, and a legal domain label; The inference chain steps include step number, inference type, premise identifier list, conclusion identifier, and truth value of the conclusion of that step.

[0010] As a further aspect of the present invention: the steps for verifying legal provisions in the first channel legal knowledge base specifically include: Iterate through the list of cited legal provisions in the intermediate representation of legal logic and the names of legal bases in the rule set, and verify each legal provision reference sequentially through the cascaded knowledge base adapter; The cascaded knowledge base adapter includes: Structured legal database adapter and local rule base adapter; Three-valued logic is used when calculating the reliability of knowledge base verification: When the legal provision exists and the matching degree is high, the verification reliability value is used; If the legal provision exists but the content does not match, a penalty value of no less than 0.5 will be applied. If no legal provision is found, a neutral value of 0.5 is used.

[0011] As a further aspect of the present invention: the steps of verifying the consistency of symbolic reasoning in the second channel answer set specifically include: Translate the intermediate representation of legal logic into an answer set program, where the translation rules include: Translate atomic legal propositions into fact predicates or negated fact predicates; Translate legal rules into implication rules; Translate exception cases into integrity constraints; Translate the reasoning chain into derivation rules; Translate the option-proposition mapping into an option support indicating predicate; Invoke an answer set solver to solve the translated answer set program, enumerate all answer sets, and detect logical inconsistency and integrity constraint violations; Extract the supported option list from the solving result, and perform consistency comparison with the options predicted by the large language model.

[0012] As a further scheme of the present invention: the steps of verifying the truth value propagation of the reasoning chain based on uncertainty theory in the third channel specifically include: Construct the premise and conclusion of each step in the reasoning chain of the intermediate representation of legal logic into proposition nodes; According to the reasoning type of each step in the reasoning chain, apply the reasoning rules of uncertain propositional logic for truth value propagation: When the reasoning type is modus ponens, given the premise truth value T(A)=a and the implication truth value T(A→B)=b: If a+b>1, then T(B)=b, and the reasoning assignment is reasonable; If a+b=1, then T(B)=min(0.5,b), and the reasoning assignment is reasonable; If a+b<1, then the reasoning assignment is pathological; When the reasoning type is modus tollens, given T(A→B)=a and T(B)=b: If a>b, then T(A)=1-a, and the reasoning assignment is reasonable; If a=b, then T(A)=max(1-a,0.5), and the reasoning assignment is reasonable; If a<b, then the reasoning assignment is pathological; When the reasoning type is hypothetical syllogism, given T(A→B)=a and T(B→C)=b: If a>0.5 and b>0.5, then T(A→C)=min(a,b), and the reasoning assignment is reasonable; If a+b≥1 and min(a,b)≤0.5, then T(A→C)=0.5, and the reasoning assignment is reasonable; If a+b<1, then the reasoning assignment is pathological; Summarize the propagation result of the reasoning chain into the truth value propagation reliability of the reasoning chain: If any step has an ill-conditioned assignment, the confidence level is 0; otherwise, the minimum of the truth values ​​of all step conclusions is taken.

[0013] As a further aspect of the present invention, the steps of uncertainty theory reliability calibration and fusion specifically include: The reliability values ​​of each verification channel used were collected: truth propagation reliability of the inference chain, knowledge base verification reliability, and procedural consistency reliability of the answer set. The procedural consistency reliability of the answer set was 0.85 when consistent and 0.4 when inconsistent. If there are compilation errors in the intermediate representation of legal logic, a compilation quality factor is introduced, with a value of max(0.4, 1.0 - 0.05 × number of compilation errors); The collected reliability factors are treated as uncertainty measures of independent uncertain events, and the uncertainty measure product axiom is applied for fusion, i.e., the calibration reliability equals the minimum value of all factors:

[0014] in, Propagate the credibility of the truth value in the inference chain. To verify the reliability of the knowledge base To answer the question of program consistency reliability, This is the compilation quality factor.

[0015] As a further aspect of the present invention: the steps of risk assessment and hallucination determination output specifically include: The initial risk score is 0, and the following risk signals are added sequentially: If the calibration reliability is lower than the dropout threshold, the risk score increases by the difference between the dropout threshold and the calibration reliability. If the response set program detects a logical inconsistency, the risk score is increased by the first weight; If there are ill-conditioning steps in the inference chain, the risk score is increased by a second weight for each ill-conditioning step; When the risk score exceeds the judgment threshold, it is determined that there is a hallucination, triggering the abandonment mechanism, marking the final answer as abandoned, and setting the credibility to 0; otherwise, the verified initial answer is output, and the credibility is taken as the calibrated reliability value. The rejection threshold, the first weight, the second weight, and the judgment threshold are preset positive numbers.

[0016] As a further aspect of the present invention: the initial reasoning and answer generation steps of the large language model further include a multi-sample voting reliability calibration sub-method based on uncertainty theory, specifically including: Based on diverse information inputs, prompts and variations, or sampling configurations, multiple independent samples are taken from the same legal clause, evidence text, or legal-related factual event statement task object. Each sampling acquires the option selection, model self-assessment confidence, and inference process. Construct a belief distribution profile, and calculate the exact set lower bound, single-option lower bound, and complementary event lower bound for each option. The exact set lower bound is the minimum confidence among all support samples for that option, and the complementary event lower bound is the minimum confidence among all samples excluding that option. Detecting contradictory options: If the self-assessed confidence levels of two or more different options all exceed the contradictory reliability threshold, then a contradiction is determined to exist; High-confidence samples are selected according to the retention ratio and weighted voting is performed. The fusion confidence of the winning option is the minimum value of the minimum confidence of its supporting samples, the consensus ratio, and the lower bound of the prediction set. If a contradiction is detected or the fusion confidence is lower than the rejection confidence threshold, this sub-method outputs a rejection flag.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention uses a legal logic compiler to transform natural language reasoning from a large language model into a formally verifiable intermediate representation. It can perform structured analysis and logical consistency checks on the processing of legal clauses, evidence texts, and statements of legal-related facts and events, thus overcoming the shortcomings of existing methods that only focus on the correctness of the answer while neglecting the reliability of the reasoning process.

[0018] By applying the propositional logic reasoning rules of uncertainty theory (affirming the antecedent, denying the consequent, and hypothetical syllogisms) to the truth propagation verification of legal clauses, evidence texts, and legal-related factual event statements, the credibility of each step in the reasoning process can be quantified. Furthermore, logical loopholes in the reasoning chain can be discovered through assignment pathology detection, a capability that existing illusion detection methods do not possess.

[0019] By employing the uncertainty measure product axiom, the reliability of three independent channels—reasoning chain verification, knowledge base verification, and symbolic reasoning—is integrated into a calibrated reliability, exhibiting stronger robustness and more comprehensive illusion coverage compared to single-channel methods. Experiments show that on a high-difficulty legal multiple-choice dataset, the compiler verification method of this invention improves accuracy by approximately 21 percentage points (from 31.33% to 52.33%), reduces the illusion rate by approximately 12 percentage points (from 50.00% to 38.33%), and improves the AIIA uncertainty honesty expression rate by approximately 24 percentage points (from 57.33% to 81.67%) compared to the baseline method.

[0020] This invention directly addresses four core court business scenarios: verification of legal provisions in judgments, application of the timeliness of new and old laws, extraction of key facts from judgment documents, and verification of substantive information in case filing registration. It meets the urgent need in judicial practice for the large language model to "never fabricate facts at the fact-checking level," enabling the large language model to be used as a rigorous and reliable legal intelligent assistant in actual judicial workflows. Attached Figure Description

[0021] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0022] Figure 1 The overall flowchart of the illusion elimination method based on the large language model in the legal field and uncertainty theory provided in the embodiments of the present invention is shown.

[0023] Figure 2 The flowchart of the legal logic compiler provided in this embodiment of the invention is shown. It shows the compilation process from natural language reasoning to the intermediate representation of legal logic.

[0024] Figure 3 This is a schematic diagram of multi-channel uncertainty theory reliability fusion provided in an embodiment of the present invention. Detailed Implementation

[0025] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] The present invention will be further explained below with reference to specific embodiments.

[0027] Please see Figures 1-3 The illusion elimination method based on the large language model in the legal field according to uncertainty theory provided in this embodiment of the invention includes the following steps: S100: Receiving and Preprocessing Legal Task Objects: This step receives the legal multiple-choice questions and parses them into a structured data representation. Specifically, the legal multiple-choice questions are parsed into legal multiple-choice question data items (LegalMCQItem), which contain the following fields: item_id: A unique identifier for the question; question: the question stem text; options: A dictionary of options, where the keys are option tags (A / B / C / D) and the values ​​are option text; correct_option: The correct option flag; correct_options: List of correct option markers (supports multiple-choice questions); jurisdiction: Legal jurisdiction, defaults to "CN" (Chinese legal jurisdiction). domain: Legal field classification, such as criminal law, civil law, administrative law, procedural law, constitutional law, and commercial law. time_sensitive: A time sensitivity indicator that identifies whether the question involves time-related legal validity judgments. authority_hints: A list of legal hints, including the names of the legal bases explicitly mentioned in the question; answer_format: The format for answering questions. "single" indicates a single-choice question, and "multi" indicates a multiple-choice question.

[0028] During the legal knowledge base retrieval phase, the system enhances the retrieval context by constructing retrieval based on legal citations in the question stem and answer choices. The specific process is as follows: S101: Use the regular expression pattern of legal citation to scan the text of the question stem and options, and identify the legal names (such as "Criminal Law", "Civil Code", "Administrative Reconsideration Law", etc.) and specific article numbers (such as "Article 24"). The regular expression pattern covers major legal categories such as the Constitution, Civil Code, Criminal Law, Criminal Procedure Law, Civil Procedure Law, Administrative Procedure Law, Administrative Reconsideration Law, Administrative Licensing Law, Company Law, Trademark Law, Patent Law, and Labor Contract Law. S102: Perform aliasing and normalization on the identified legal names, for example, normalize "Civil Law" to "Civil Code"; S103: Sequentially retrieve the full text of matching legal provisions through the cascading knowledge base adapters. The cascading order is as follows: First, query the Structured Law Database Adapter, which loads an offline full-text legal database (22,552 legal records from flk.npc.gov.cn), quickly locates the target legal record using an inverted index of legal names, and then extracts the specific content of the provisions using regular expressions for article splitting; if no match is found in the structured legal database, query the Local Rule Base Adapter, which contains pre-set basic legal provisions in the fields of criminal law (8 core provisions), civil law (3 provisions), administrative law (2 provisions), constitutional law (3 provisions), procedural law (2 provisions), and commercial law (2 provisions); S104: Format the full text of the retrieved legal provisions into the format "[Legal Provision Citation Name] Legal Provision Original Text", with a maximum of 5 legal provisions, and concatenate them into a search enhancement context string, which is then injected into the subsequent large language model prompt template.

[0029] S200: Initial Inference and Response Generation for Large Language Models This step utilizes a large language model to perform initial reasoning on the legal multiple-choice questions, generating structured answers that include option selections, the reasoning process, and the model's self-assessed confidence level. The specific process is as follows: S201: Construct a prompt template for legal multiple-choice questions. Inject the question stem text, option text, and retrieval enhancement context obtained in S100 into the prompt template. The prompt template requires the model to output the following content in a specified format: option selection, selected option list, option confidences, answer explanation text, overall confidence (value 0-1), reasoning process, and cited legal provisions. S202: Call the large language model with a fixed temperature parameter (temperature=0.1) to obtain the original text response; S203: Parse the original response, extract the structured answer fields in JSON format, and construct a ModelAnswer object, which includes the method name, option selection, option self-rating confidence, overall self-rating confidence, reasoning process, original response text, and metadata.

[0030] In a preferred embodiment of the present invention, the initial inference of the large language model further includes multi-sample voting reliability calibration based on uncertainty theory (uncertainty_theory method), the specific process of which is as follows: S211: Based on diverse information input, prompt expression transformation, or sampling configuration, perform multiple independent samplings (3 times by default) on the same legal clause, evidence text, or legal-related factual event statement task object. Each sampling acquires the selection of options, model self-evaluation of confidence, and inference process, and constructs a sampling trajectory list. S212: An adaptive early termination mechanism is introduced during the sampling process: After reaching the minimum number of preheating samples (default 3 times), the consensus ratio is calculated after each sampling. When the consensus ratio reaches the consensus threshold (default 0.75), the sampling is terminated early to avoid unnecessary model call overhead. S213: Construct a belief profile and calculate lower bounds for three uncertainty measures for each option label: Exact set lower bound: The exact set lower bound for option O is defined as the minimum confidence level among all sampled trajectories where this option is selected, i.e.:

[0031] Singleton lower bound: The singleton lower bound for option O is defined as the minimum confidence level among all sampled trajectories where exactly only that single option is selected, i.e.:

[0032] The complement lower bound is defined as the minimum confidence level among the sampled trajectories of all complements of option O that have selected O.

[0033] Where U is the set of all option tags; S214: Detect conflicting options: Determine if there is a conflict among the retained high-confidence samples. If the self-assessed confidence of two or more different options exceeds the conflict confidence threshold (default 0.6), then a conflict is determined to exist. S215: Calculate fusion reliability: Perform a weighted vote on the high-reliability samples truncated from the retention ratio (default 0.6), and the fusion reliability of the winning option is the minimum value of the following three:

[0034] In equations (1) to (4), Indicates the candidate options to be evaluated; Indicates the first The reasoning trajectory obtained from sub-independent sampling; Indicates the first The model's self-assessment reliability corresponding to each sampling; This indicates a single option for the output of the sampling trajectory; This represents the set of options selected for the sampling trajectory; Represents the set of all candidate options; This indicates the minimum confidence level in the sample that supports the winning option; Indicates the weighted consensus ratio of the winning option; Indicates the option that wins the weighted vote; It represents the lower bound of the exact set of winning options.

[0035] S216: If a contradiction is detected or the fusion confidence is lower than the dropout confidence threshold (default 0.55), then a dropout flag (ABSTAIN) is output and the confidence is set to 0.

[0036] S300: Legal Logic Translation: This step is one of the core innovations of this invention: "compiling" the natural language reasoning of the large language model into a structured legal logic intermediate representation (Legal Logic IR). The specific process is as follows: S301: Constructing Legal Formal Logic Compilation Hints. The compilation hint template contains three parts of input: the question stem text and option text, the model's option selection and reasoning process, and the extraction task instructions. The extraction task instructions require the model to decompose its reasoning into the following five structured components: (1) Propositions: Atomic legal propositions involved in the reasoning. Each proposition contains a unique identifier prop_id (P1, P2, ...), proposition text, predicate (e.g., "constituting a crime and suspending it"), subject (e.g., "the defendant"), truth_value (0-1 range, representing the model's self-reported confidence level of the proposition's validity), source ("proposition stem" / "legal provision" / "reasoning") and negated flag. (2) Rule list: Legal rules that are cited or implied. Each rule contains a unique identifier rule_id (R1,R2,...), the legal basis name authority (e.g., "Article 24 of the Criminal Law"), the rule text, a list of antecedent proposition identifiers if_clauses, a consequent proposition identifier then_clause, a list of exceptions exceptions, and a legal domain label domain; (3) Reasoning chain: A sequence of reasoning steps arranged in order. Each step includes step number step_id, inference type ("modus_ponens" / "modus_tollens" / "hypothetical_syllogism" / "direct_application" / "elimination"), premise ID list premise_ids, conclusion proposition ID conclusion_id, truth value of the conclusion of this step truth_value, and supplementary notes; (4) List of cited legal provisions (cited_authorities): A list of all legal bases cited; (5) Option-proposition map: A list of supporting / opposing propositions for each option.

[0037] S302: Using a temperature parameter of 0.0, call the verification-specific large language model to execute compilation prompts and obtain JSON format output; S303: Parse the compiled output into JSON, and parse the raw JSON data into lists of LegalProposition, LegalRule, and ReasoningStep data structures respectively; S304: Perform semantic verification on the parsed intermediate representation of legal logic: (a) Traverse the list of premise identifiers for each step in the reasoning chain, and check whether each premise identifier exists in the identifier set of the proposition set or the rule set. If it does not exist, record a compilation error. (b) Check if the conclusion identifier for each step exists in the proposition set; if not, record a compilation error. (c) Check if the truth value of each step is within the range [0,1]. If it is outside the range, truncate it to [0,1] and record the compilation error. (d) Check if the antecedent proposition identifier of the rule exists in the proposition set; if not, record a compilation error. (e) If the list of propositions is empty, record a compilation error "no propositions extracted".

[0038] S400: Multi-channel form verification: This step performs formal verification on the intermediate legal logic representation generated by S300 through three independent channels. The three channels are independent of each other and can be executed in parallel.

[0039] S410: First Channel – Legal Knowledge Base Verification: This channel verifies the authenticity and accuracy of all legal provisions cited in the intermediate representations of legal logic. The specific process is as follows: S411: Iterate through the `authority` field in the `cited_authorities` list and `rules` list in the middle representation, and collect all unique legal citations; S412: Each legal citation is verified using a cascading knowledge base adapter (CompositeKBAdapter). The cascading order is as follows: The first layer is the StructuredLawDBAdapter: (i) The adapter adopts the singleton pattern and loads an offline full-text legal database on the first access. This database contains 22,552 legal records obtained from the National Legal Database (flk.npc.gov.cn); (ii) After loading, construct a forward index of titles and an inverted index of short titles for each law, and extract the article-level content of the law text through article splitting regular expressions (matching the format "Article X"); (iii) Parse the legal citations, extract the legal names and article numbers, locate the target legal records through the short heading inverted index, and select the latest version with the status of "valid"; (iv) If an exact match is found, return the verification result: exists=True, confidence=0.95, along with the full text of the matched clause; if a law is found but the specific clause is not extracted, return confidence=0.75; if only the law name is found without specifying the clause number, return confidence=0.85. The second layer, the LocalRuleBaseAdapter, contains 20 pre-built core legal provisions, covering criminal law (criminal abandonment, criminal preparation, attempted crime, joint crime, necessity, corporate crime, application of the death penalty, non-application of the death penalty), civil law (conditions for the validity of acts, gift contracts, contracts for work), administrative law (administrative review, administrative licensing), constitutional law (equality of citizens, special administrative regions, private property), procedural law (final statement, jurisdiction by agreement), and commercial law (shareholders' meeting resolutions, trademark transfer). For exact matching, confidence = 0.85. S413: Calculate the reliability of knowledge base verification Three-valued logic is used to distinguish between three states: "verified," "unverifiable," and "contradictory."

[0040] Where N is the total number of legal provisions verified, and the value of score_i is determined according to the following rules: If the legal provision exists and confidence ≥ 0.7 (verified): score_i = confidence_i; If the legal provision exists but confidence < 0.7 (partial match): score_i = max(0.5, confidence_i); If the legal provision does not exist (cannot be verified): score_i = 0.5; Here, score_i is 0.5 instead of 0 when the legal provision does not exist, which reflects the essential difference between "not knowing" and "contradiction" in the uncertainty theory—unverifiable only indicates a gap in the knowledge base coverage and should not be equated with denying the correctness of the legal provision.

[0041] S420: Second Channel – Consistency Verification of Symbolic Reasoning in the Response Set Program: This channel translates the intermediate representation of legal logic into an Answer Set Program (ASP), using non-monotonic logical reasoning to formally verify the logical consistency of legal clauses, evidentiary texts, and the processing of statements of legal-related facts and events. The specific process is as follows: S421: Translate the legal logic intermediate representation into an ASP program. The translation rules are as follows: (a) Proposition Translation: For each atomic legal proposition, if negated=False, it is translated into the fact holds(predicate,subject); if negated=True, it is translated into the fact neg(predicate,subject). The predicate and subject are converted into valid ASP atomic names using an identifier cleansing function (removing non-ASCII characters, and mapping CJK characters to stable suffixes via MD5 hashing). (b) Rule translation: For each legal rule, the antecedent proposition is combined (connected by commas) to form the rule body, and the consequent proposition is the rule head, which is translated into ASP implication rule "holds(head,subject):-holds(body1,s1),holds(body2,s2),..."; the exceptions to the rule are translated into integrity constraints ":-holds(head,subject),holds(exception,subject)."; (c) Translation of the reasoning chain: For each step in the reasoning chain, the relationship between the premise proposition and the conclusion proposition is translated into a derived rule; (d) Option map translation: For each option in option_proposition_map, the list of supporting propositions is translated into the option support indicator predicate "option_supported(opt):-holds(p1,s1),holds(p2,s2),...". (e) Output instructions: Add the #show directive to output the three types of predicates: holds / 2, neg / 2, and option_supported / 1; S422: Call the Clingo answer set solver (models=0 mode, enumerating all answer sets) to solve the translated ASP program: If the solution result is satisfactory (satisfiable=True), then consistent=True, and record the list of atoms for all answer sets; If the solution result is unsatisfiable (satisfiable=False), then consistent=False, and a logical inconsistency conflict is recorded; When Clingo is not installed, the system automatically switches to fallback analysis mode, which performs approximate consistency checks by analyzing facts, rules and integrity constraints in the ASP program text through pattern matching. S423: Extract the list of options supported by the predicate option_supported from the first answer set, compare it with the options predicted by the large language model, and record the consistency deviation if the predicted option is not in the support list.

[0042] S430: Third Channel – Truth Propagation Verification of Uncertainty Theory Inference Chain: This channel represents the core theoretical innovation of this invention. It applies the rules of indeterminate propositional logic reasoning from Chapter 8 of Professor Liu Baoding's Uncertainty Theory at Tsinghua University to the truth propagation verification of legal clauses, evidentiary texts, and legally relevant factual event statements. The specific process is as follows: S431: Construct a Proposition Node for each step in the reasoning chain. For each step's premise identifier, search for the corresponding object in the proposition set and rule set, extract its truth value, and construct a PropositionNode. If the premise is an implication proposition (A→B), then the truth value T(A→B) = max(1-T(A),T(B)) (based on the definition of truth value for uncertain implication). The conclusion proposition is also constructed as a PropositionNode. S432: Based on the reasoning type at each step, select the corresponding uncertain reasoning rule for truth propagation: (a) Uncertain Modus Ponens: Based on Formula (8.54) in Section 5 of Chapter 8 of Professor Liu Baoding's "Uncertainty Theory" 5th Edition (2025) from Tsinghua University.

[0043] Given the truth value of the premise proposition T(A) = a and the truth value of the implied proposition T(A→B) = b, derive the truth value of the conclusion proposition T(B): Rule 1: If a + b > 1, then T(B) = b, the inference assignment is reasonable (well_assigned=True); Rule 2: If a + b = 1 (with numerical precision ε = 10) -9 If the value is within the range of 0.5, then T(B) = min(0.5, b), and the inference assignment is reasonable. Rule 3: If a + b < 1, then the inference assignment is ill-conditioned (well_assigned = False), and T(B) cannot be determined.

[0044] The truth value combination assigning a pathological representation premise does not support effective reasoning via modus ponens, which in the processing of legal provisions, evidence texts, and legal-related factual event statements means that the credibility of the premise is insufficient to support the conclusion.

[0045] (2) Uncertain Modus Tollens: Based on Formula (8.57) in Section 6, Chapter 8 of the 5th edition (2025) of *Uncertainty Theory* by Professor Baoding Liu from Tsinghua University.

[0046] Given the truth value of the implicative proposition T(A→B)=a and the truth value of the consequent proposition T(B)=b, derive the truth value of the antecedent proposition T(A): Rule 1: If a>b, then T(A)=1-a, and the reasoning assignment is valid; Rule 2: If a=b (within the precision ε range), then T(A)=max(1-a, 0.5), and the reasoning assignment is valid; Rule 3: If a<b, the reasoning assignment is pathological, and T(A) cannot be determined.

[0047] (3) Uncertain Hypothetical Syllogism: Based on Formula (8.60) in Section 7, Chapter 8 of the 5th edition (2025) of *Uncertainty Theory* by Professor Baoding Liu from Tsinghua University.

[0048] Given T(A→B)=a and T(B→C)=b, derive T(A→C): Rule 1: If a>0.5 and b>0.5, then T(A→C)=min(a, b), and the reasoning assignment is valid; Rule 2: If a+b≥1 and min(a,b)≤0.5, then T(A→C)=0.5, and the reasoning assignment is valid (in this case, the information is insufficient to determine the truth value of the transitive implication, so the value 0.5 is assigned according to the maximum uncertainty principle); Rule 3: If a+b<1, the reasoning assignment is pathological, and T(A→C) cannot be determined.

[0049] Rule 2 therein embodies the maximum uncertainty principle of uncertainty theory: when the truth value is not uniquely determined, a value as close to 0.5 as possible is assigned.

[0050] S433: The truth value propagation is executed sequentially along the reasoning chain, and the conclusion truth value of each step is updated to the conclusion proposition node when the assignment is valid, to be used as a premise for subsequent steps (feedforward propagation); S434: Aggregate the propagation result of the reasoning chain into the reliability of truth value propagation of the reasoning chain : If the reasoning chain is empty (no reasoning steps). (Triggered the abandonment protection); If any step has an ill-conditioned assignment (well_assigned=False). (This indicates a logical break in the reasoning chain, directly triggering a low-confidence signal.) Otherwise, calculate the truth propagation confidence of the inference chain according to the following formula: ;

[0051] This formula embodies the conservative principle in uncertainty theory that "the weakest link determines the overall credibility."

[0052] In other embodiments of the present invention, the multi-channel formal verification may include only any two of the three channels mentioned above, as long as it includes at least the answer set procedural symbolic reasoning consistency verification channel, thus achieving effective verification of the consistency of reasoning logic. A complete combination of the three channels is the preferred solution, while a combination of two channels can also achieve part of the inventive objectives of the present invention, and both fall within the protection scope of the present invention.

[0053] S500: Uncertainty Theory Reliability Calibration and Fusion This step integrates the reliability scores of the three independent validation channels into the final calibration reliability using the uncertainty measure product axiom. The specific process is as follows: S501: Collect reliability values ​​for three independent channels: The truth propagation calibration reliability of the inference chain calculated by S430. If... This was adjusted to 0.5 (neutral value) to avoid extreme values ​​in a single channel directly negating positive evidence from other channels; The knowledge base verification reliability calculated by S410; The consistency reliability of the S420 response set is 0.85 when consistent=True and 0.4 when consistent=False. S502: If compilation errors exist in the intermediate representation of legal logic (compilation_errors is not empty), introduce a compilation quality factor:

[0054] Where |compilation_errors| represents the number of compilation errors. This factor incorporates compilation quality into the reliability assessment; the more compilation errors, the lower the reliability, but it is guaranteed to be no less than a lower bound of 0.4. S503: Apply the axiom of product of uncertain measures (Axiom 4) to perform multi-channel uncertainty theory reliability fusion.

[0055] The mathematical formulation of the axiom of the product of uncertain measures (Axiom 4, Chapter 2, of Professor Liu Baoding's "Uncertainty Theory," 5th edition (2025), Tsinghua University) is as follows: Let Λ_1, Λ_2, ..., Λ_n be events defined on the independent uncertainty space (Γ_1, L_1, M_1), (Γ_2, L_2, M_2), ..., (Γ_n, L_n, M_n). Then the product uncertainty measure is:

[0056] in This indicates the minimum value operation.

[0057] Applying this axiom to the multi-channel uncertainty theory reliability fusion scenario of this invention: considering inference chain verification, knowledge base verification, and ASP consistency verification as events on three independent uncertainty spaces, and the reliability value of each channel as the uncertainty measure of the corresponding event, then the joint reliability of the three channels is equal to the minimum value of the reliability of each channel:

[0058] If a compilation quality factor exists, then:

[0059] The rationale for choosing the product axiom (minimum value) over the multiplication rule of probability theory (product) is as follows: Within the framework of uncertainty theory, each channel reflects the degree of belief in the event that the answer is credible, rather than frequency probability. Belief degree depends on personal knowledge rather than a large number of repeated trials; the channels are cognitively independent rather than statistically independent. The product axiom ensures that the final reliability does not exceed the reliability of any single channel, i.e., the "weakest link effect"—the weakest validating channel determines the upper bound of the overall reliability.

[0060] S600: Risk Assessment and Hallucination Determination Output: This step integrates calibration reliability and multidimensional risk signals to make the final determination of hallucination. The specific process is as follows: S601: Initialize the risk score R=0, and then sequentially add the following risk signals: Signal 1 (insufficient confidence): If (If the threshold for abandoning answers is reached), then R increases. The default threshold for dropping an answer is 0.55. Signal 2 (Logical Inconsistency): If ASP detects logical inconsistency (consistent=False), then R increases by the first weight, i.e., 0.3; Signal 3 (Ill-conditioned Reasoning): Counts the number of steps in the reasoning chain that are assigned ill-conditioned values. ,like Then R is given a second weight, i.e. The above risk score calculation can be expressed as equation (5); S602: Hallucination Detection: If R is greater than the detection threshold, i.e., R>0.5, a hallucination is detected, and the following operations are performed: Mark the final answer option as "ABSTAIN" (discard answer); Clear the list of selected options; Set the final confidence level to 0.0; Set the inference specification to "compiler verification triggered ABSTAIN"; If R≤0.5, it is determined that there is no hallucination (or the risk of hallucination is acceptable), the initial answer's option selection and reasoning process are retained, and the credibility is set to C_calibrated; S603: Construct a complete compilation verification result record, including: intermediate representation of legal logic, list of legal provision verification results, ASP solution results, truth value sequence of inference chain, calibration reliability, formal consistency flag, knowledge base verification flag, inference validity flag, and verification notes; S604: Perform six-dimensional hallucination signal detection on the final output according to the AIIA's "Evaluation Framework for Illusions in Generative Artificial Intelligence Models": Dimension 1 – Factual Accuracy: Compare the selected options of the final answer with the correct options, and calculate the exact match accuracy and the F1 score of the option set; Dimension 2 – Source Fidelity: Check whether the kb_verified flag in the compilation verification result is False (knowledge base verification found a contradiction), and whether the cited legal provisions exceed the scope of the legal provisions hinted at in the question; Dimension 3 – Internal Consistency: Check whether the consistency flag in the model's answer metadata is True (indicating high-confidence contradictory options in multi-sample sampling); Dimension 4 – Instruction Compliance: Check whether the output option markers are within the set of valid options for the question; Dimension 5 – Logical Reasoning: Check the formal consistency of the ASP and the validity of the reasoning chain. If either is False, the logical reasoning is marked as incorrect. Dimension 6 – AIIA Uncertainty Honest Expression: Checks whether the self-rated confidence level exceeds the overconfidence threshold (default 0.8) when the answer is wrong. If so, it is marked as uncertainty dishonesty. This dimension is an evaluation indicator, which is different from the uncertainty theory mathematical mechanism used for reliability calibration in this invention. S605: Severity grading based on six-dimensional detection results: If you choose not to answer: the severity level is "none"; If the answer is correct and all six dimensions are normal: the severity is "none"; If the answer is correct but some dimensions are abnormal: the severity is "minor"; If the answer is incorrect, the source is unreliable, and there are factual errors: the severity is "fatal"; If the answer is incorrect and the self-rated confidence level exceeds the overconfidence threshold: the severity is "serious"; Other incorrect answers: severity level is "moderate".

[0061] The experiment verified the following: To verify the effectiveness of the method of this invention, a systematic experiment was conducted based on the LexEval legal benchmark dataset. The experiment covered legal multiple-choice question datasets of different difficulty levels, and compared the performance of 13 methods. The experimental data is only used as one implementation method to verify the effect of the embodiments of this invention, to illustrate the auxiliary role of different verification channels in hallucination risk identification, severe hallucination suppression, and answer abandonment control, and does not constitute a limitation on the scope of protection of this invention or the applicable data scale.

[0062] The experimental environment and dataset are as follows: The experiment used the following configuration: Main inference model: DeepSeek series large language model, called through two API endpoints: SiliconFlow and DeepSeek; Validation Model: A standalone DeepSeek validation model used for legal logic compilation and answer verification; Legal knowledge base: In this experiment, a structured database containing 22,552 legal and regulatory records was constructed based on the twang2218 / chinese-law-and-regulations dataset; in other implementations, the number of database records can be adjusted according to data source updates. Symbolic reasoning engine: Clingo 5.x answer set solver; The experimental dataset consists of four sets. Confirmed through court practice research, each dataset corresponds to a real judicial business scenario, as shown in Table 1. Table 1 Experimental Dataset:

[0063] The comparison method is as follows: The experiment compared the following 13 methods: (1) baseline: The pure model directly answers without retrieval enhancement; (2) direct: a direct answer with enhanced search context; (3) Self-consistency: Multiple sampling and voting to select the high-frequency answer; (4) Verifier: Uses an independent verification model to review the initial answer; (5) uncertainty_theory: Multi-sample voting reliability calibration based on uncertainty theory; (6) aggregate_judge: The evaluation model makes a comprehensive decision after self-consistent sampling; (7) pipeline_no_uncertainty: a pipeline with direct response, self-consistency, and a validator for three-way evaluation; (8) pipeline_with_uncertainty: The above pipelines are supplemented with uncertainty theory as the fourth candidate; (9) compiled_verification: The core method of this invention, legal logic compilation + three-channel formal verification + uncertainty confidence fusion; (10) pipeline_with_compiler: a complete five-candidate evaluation pipeline that integrates compilation verification as the fifth candidate; (11) calibrated_uncertainty_only: Ablation experiments - calibration is performed using only uncertainty theory, without formal logic verification; (12) calibrated_formal_only: ablation experiment - using only IR compilation + ASP + KB verification, without using uncertain theory inference chain propagation; (13) calibrated_combined: ablation experiment - a complete combination of uncertainty theory and formal logic verification.

[0064] Experiment Result 1: High-Difficulty Multiple Choice Question Dataset (Dataset 3, 300 Questions): This dataset contains multiple-choice questions on legal evolution and institutional comparison, which are the most difficult and best demonstrate the advantages of the method proposed in this invention. The main experimental results are shown in Table 2.

[0065] Table 2 Performance comparison of various methods on a high-difficulty multiple-choice question dataset:

[0066] As can be seen from Table 2: (1) The compiled_verification method of the present invention achieves an accuracy of 52.33%, which is about 21 percentage points higher than the baseline (31.33%→52.33%). It is the best among all other methods, demonstrating the significant effect of legal logic compilation and three-channel formal verification. (2) Regarding the hallucination rate, the compiled_verification rate was 38.33%, which was about 12 percentage points lower than the baseline of 50.00%. More importantly, the serious hallucination rate dropped from 42.67% to 18.33%, a decrease of 24 percentage points, indicating that the method of the present invention can effectively identify and suppress highly credible wrong answers. (3) The AIIA uncertainty honest expression rate increased from 57.33% in the baseline to 81.67% in compiled_verification, an increase of about 24 percentage points. This indicates that the method of the present invention can more honestly express uncertainty (reduce credibility or abandon the answer) when the model gives a wrong answer, rather than showing false high credibility on wrong answers; (4) It is worth noting that the illusion rates of the self_consistency and verifier methods (59.00%, 68.33%) are actually higher than the baseline (50.00%). This is because the voting mechanism and the verifier tend to amplify systematic biases in multiple-choice scenarios. When the model consistently makes the same mistake, multiple sampling and verification not only fail to correct the error but also enhance the apparent credibility of the wrong answer, leading to a surge in overconfidence (57.33%, 65.00%). This is precisely the motivation for introducing formal logic verification in this invention.

[0067] Experimental Result 2: Simple Multiple Choice Question Dataset (Dataset 1, 1000 questions): This dataset was used to verify whether the method of this invention introduces the side effect of over-conservatism in simple scenarios. That is, for questions that the model can already answer stably, compilation verification should not sacrifice usability by throwing out a large number of answers. The main experimental results are shown in Table 3.

[0068] Table 3 Performance comparison of various methods on a simple multiple-choice question dataset:

[0069] As can be seen from Table 3: (1) On simple datasets, the uncertainty_theory method achieves 100% accuracy, indicating that multi-sample voting of uncertainty theory has perfect consensus on simple problems; (2) The compiled_verification method achieved an accuracy of 97.90% with a hallucination rate of 0 and a dropout rate of 2.10%, indicating that in simple scenarios, compiled verification can achieve a lower hallucination risk with a smaller dropout cost, and there was no situation where a large number of dropouts were used to improve the metrics. (3) The hallucination detection F1 of compiled_verification reached 90.91%, and that of calibrated_formal_only reached 97.56%, indicating that the formal logic verification channel has extremely high hallucination detection accuracy on simple datasets.

[0070] Experimental Result 3: Ablation Experiment: To verify the respective contributions of the uncertainty theory channel and the formal logic channel, and the necessity of their fusion, a systematic ablation experiment was designed on multiple datasets. First, benchmark ablation was performed on a large, simple dataset (dataset 1, 1000 questions), and the results are shown in Table 4.

[0071] Table 4 Comparison of ablation experiments (Dataset 1, 1000 questions):

[0072] To further verify the stability of the ablation conclusions, ablation experiments were repeated on datasets of different difficulty levels, and the results are shown in Table 4-1.

[0073] Table 4-1 Comparison of ablation experiments with varying difficulty:

[0074] Note: Dataset 1 is easy (1000 questions), Dataset 2 is medium (100 questions), and Dataset 4 is easy (approximately 496 questions).

[0075] On the highly challenging dataset (Dataset 3, 300 multiple-choice questions), the ablation experiments presented extreme but highly illustrative results, as shown in Table 4-2: Table 4-2 Comparison of Ablation in High-Difficulty Stress Tests (Dataset 3, 300 Questions):

[0076] The ablation experiments revealed the following key conclusions: (1) Complementary advantages of multi-channel calibration: In the large sample benchmark ablation (Table 4), calibrated_uncertainty_only achieved an accuracy of 99.10%, calibrated_formal_only achieved an accuracy of 97.90%, and calibrated_combined achieved an accuracy of 99.80% after fusion—higher than any single channel, and the rejection rate was only 0.20% (lower than the 0.90% and 2.10% of the two single channels), indicating that multi-channel calibration has a complementary effect on this dataset; (2) The vulnerability of a single channel increases with difficulty: Cross-difficulty ablation (Table 4-1) shows that the uncertain channel exhibits extreme degradation of 100% rejection rate on dataset 2 (unstable consensus rate leads to frequent rejections triggered by contradiction detection and low confidence thresholds), while the formal logic channel exhibits extreme degradation of 100% rejection rate on dataset 3 (Table 4-2). The "collapse intervals" of the two channels do not overlap—the uncertain channel collapses at medium difficulty, and the formal logic channel collapses at high difficulty—this indicates that a single calibration channel may have a failure interval under different task difficulties, thus supporting the necessity of introducing multi-channel cross-validation and unified risk assessment; (3) The robustness of the complete method far exceeds that of the ablation combination: In the high-difficulty stress test (Table 4-2), all three ablation methods failed (accuracy ≤ 19%), but the complete compiled_verification method still achieved an accuracy of 52.33%, with only 9.33% of responses being rejected. This indicates that compiled_verification is not a simple superposition of uncertain channels and formal logic channels, but rather achieves effective overcoming of single-channel failure scenarios through the synergistic effect of legal logic compilation, multi-channel cross-validation, and uncertain confidence fusion.

[0077] Experimental Result 4: Robustness Validation Across Datasets To verify the robustness of the method of this invention on different difficulty levels and different task types, the performance of the compiled_verification method and the baseline on four datasets were compared horizontally, and the results are shown in Table 4-3.

[0078] Table 4-3 Comparison of robustness of compiled_verification across datasets:

[0079] As can be seen from Table 4-3: (1) On simple datasets (dataset 1, dataset 4) and medium datasets (dataset 2), the accuracy of compiled_verification is basically the same as the baseline (fluctuating within ±1 percentage point), indicating that the method of the present invention maintains overall usability on questions that the model can answer correctly, and does not show a significant tendency to overcorrect. (2) On the highly challenging dataset (dataset 3), compiled_verification showed the most significant advantage: the accuracy jumped from 31.33% to 52.33% (an increase of 21 percentage points), while the hallucination rate decreased from 50.00% to 38.33% (a decrease of 12 percentage points). This indicates that the value of the method in this invention is mainly reflected in difficult task scenarios where large language models are prone to hallucinations—precisely the scenarios in legal practice where the risk of hallucinations is most critical to prevent. (3) Dataset 4 (case entity information extraction, about 496 questions) was used as a reference group to verify the stability of the method on a basic task with high accuracy: on a simple task where the baseline had reached a high accuracy of 99.60%, compiled_verification maintained an accuracy of 98.58%, the severe hallucination rate dropped to 0.00%, and the AIIA uncertainty honest expression rate reached 100.00%, further confirming the safety of the method of the present invention.

[0080] Case Analysis: The following three typical cases illustrate the process of hallucination detection and elimination using the method of the present invention.

[0081] Case 1: Compile, verify, and correct baseline errors (Problem 1_3-1): Question: Which of the following statements about the evolution of law are correct? (Multiple choice, correct answers are CD) Baseline method: Select C, model self-rated confidence level 0.90, judgment result is wrong (only C in CD was selected, D was missed), hallucination severity is "serious" - the model gave an incomplete answer with a high self-rated confidence level of 0.90, which is a typical case of overconfidence hallucination.

[0082] The compiled_verification method selects CD, with a confidence level of 0.50, and the result is considered correct. This invention's method decomposes the model reasoning into propositions and reasoning chains through legal logic compilation. The truth propagation of the uncertain theoretical reasoning chain reveals that the truth propagation chain supporting the proposition in option D is reasonable (well_assigned=True). ASP symbolic reasoning also supports option D. Therefore, the compilation verification corrects the baseline omissions and outputs the correct answer with a conservative confidence level of 0.50.

[0083] On the same question, `self_consistency` (self-rated confidence 0.90), `verifier` (self-rated confidence 0.90), and `uncertainty_theory` (ABSTAIN) all failed to provide the correct answers C and D. `self_consistency` and `verifier` output the incorrect answer C with high self-rated confidence, while `uncertainty_theory` simply abandoned the question. Only `compiled_verification` and `pipeline_with_compiler` correctly provided C and D.

[0084] Case 2: Compiling and verifying to find the correct answer in the discrepancy (Question 1_3-3): Question: Which of the following statements about the evolution of law are correct? (Multiple choice, correct answers are B and D) Baseline method: Select BC, self-rated confidence level 0.80, error - selected B and C instead of B and D.

[0085] Direct method: Select BC, self-rated confidence level 0.90, error - higher self-rated confidence level but the same error.

[0086] The compiled verification method: BD was selected, with a confidence level of 0.85, which is correct. Through legal logic compilation, the legal knowledge base verification stage retrieved the relevant original legal provisions from the structured legal database. It was found that the legal basis of option C deviated from the actual provisions (the modification history described in option C was inconsistent with the legal records in the database), while the description of option D matched the original legal provisions. ASP symbolic reasoning confirmed the logical consistency of the reasoning path in option D. The three-channel fusion yielded a calibration confidence level of 0.85.

[0087] Case 3: Partial Correction of Compiler Verification on Extremely Difficult Problems (Problems 1_3-4): Question: Which of the following statements about the evolution of law are correct? (Multiple choice, correct answers are ACD) Baseline method: Select D, self-rated confidence level 0.70, incorrect - only one of the three correct options was selected.

[0088] The compiled_verification method: Option C and D were selected with a confidence level of 0.85, partially correct (F1 = 0.80, although not completely correct, the F1 score improved from 0.50 in the baseline to 0.80). Compiled verification confirmed through truth propagation of the reasoning chain that the supporting propositions of options C and D had reasonable truth values. However, the truth propagation of the supporting proposition for option A had a boundary condition of a one-step hypothetical syllogism (a+b≈1, min(a,b)≈0.5), resulting in a lower confidence level for the truth propagation of the reasoning chain for A, which prevented it from being included in the final selection. Nevertheless, compared to the baseline which missed two correct options, A and C (F1 = 0.50), compiled_verification only missed option A (F1 = 0.80), demonstrating the incremental error correction capability of compiled verification.

[0089] AIIA's comprehensive evaluation across six dimensions: Table 5 shows a comparison of the six-dimensional achievement rates of various methods on the high-difficulty dataset (dataset 3) according to the six-dimensional evaluation system of AIIA’s “Illusion Evaluation Framework for Generative Artificial Intelligence Model Applications”.

[0090] Table 5 Comparison of AIIA Six-Dimensional Completion Rates (Dataset 3, 300 Questions):

[0091] As can be seen from Table 5: (1) compiled_verification significantly outperformed the AIIA uncertainty honesty assessment in two dimensions. Factual accuracy is a core requirement for processing legal clauses, evidence texts, and legal-related factual events. An accuracy of 52.33% means that compiled verification can help the model select approximately 21% more correct answers in multiple-choice questions. AIIA uncertainty honesty assessment is one of the illusion assessment dimensions, used to evaluate whether the model expresses uncertainty in a low-confidence or abandoned manner in erroneous or low-reliability outputs. An accuracy of 81.67% means that the model has a higher probability of expressing uncertainty (low confidence or abandoned answer) when giving incorrect answers, rather than giving false high confidence. (2) The compliance rate of compiled_verification in the logical reasoning dimension is 84.67% instead of 100.00%. This is because compiled verification introduces two additional logical checking channels: ASP symbolic reasoning and truth propagation of the reasoning chain. These channels can detect logical inconsistencies that the baseline method cannot find (the logical reasoning of the baseline seems to be 100% compliant, but in fact it is because no form of logical checking is performed). Therefore, the 84.67% of compiled_verification reflects a more rigorous and realistic assessment of logical reasoning quality. (3) All methods achieved 100.00% in terms of instruction compliance, source fidelity and internal consistency, indicating that the method of the present invention does not have any negative impact on the output format standardization, source citation fidelity and answer internal consistency while introducing multi-channel formal verification.

[0092] Distribution of hallucination severity: According to Chapter 8, "Assessment Result Recognition," of the AIIA's "Evaluation Framework for Illusions in Generative Artificial Intelligence Model Applications" (AIIA / T 0294-2026, released in April 2026), the severity of illusions is divided into four levels: mild, moderate, severe, and fatal. The core distinguishing criteria in the original AIIA document are as follows: Mild illusion refers to "non-critical detail errors that do not affect the accuracy of core information and will not mislead users"; moderate illusion refers to "errors in key information, but the errors are easily identifiable"; severe illusion refers to "fictional or severely distorted core facts, and the erroneous content is difficult to identify"; fatal illusion refers to "complete fabrication or fundamental errors that lead to safety, legal, or personal injury."

[0093] This invention addresses the automated evaluation scenario of legal multiple-choice questions by operationalizing the aforementioned AIIA definition into the following quantifiable judgment rules: Normal (none): The model abandons the answer (honestly expressing uncertainty), or the answer is correct and no hallucination signals are triggered (unfaithful source, internal contradiction, instruction violation, and reasoning error are not detected) - This level is an additional addition outside the AIIA framework to identify outputs without hallucinations; Minor: The answer is correct, but there are non-critical anomaly signals (such as boundary condition deviations detected by truth propagation in the inference chain of compilation verification, or non-core logic deviations found by ASP symbolic inference), which corresponds to the definition of "not affecting the accuracy of core information" in AIIA minor illusion; Moderate: The answer is incorrect, but the credibility of the model output is low (below the overconfidence threshold θ=0.8), which corresponds to the definition of "errors are easy to identify" in AIIA's moderate illusion - low credibility means that the model shows uncertainty about the answer, and such errors are easy to identify and correct in the downstream manual review process; Serious: The answer is wrong and the confidence level of the model output is ≥0.8, which corresponds to the definition of "error is hard to identify" in AIIA serious illusion - high confidence makes the wrong answer seem reliable, making it difficult for downstream reviewers to detect, and it is most harmful in legal scenarios; Fatal: The answer is wrong, and there are also signs of factual error and dishonesty of source (i.e. the model fabricates non-existent legal basis to support the wrong answer), which corresponds to the definition of "causing legal harm" in AIIA Fatal Illusion.

[0094] The "hallucination rate" is the sum of the percentages of moderate, severe, and fatal levels. Mild levels are not included in the hallucination count because they do not affect the correctness of the answer. The distribution of hallucination severity for each method on the high-difficulty dataset is shown in Table 6.

[0095] Table 6 Distribution of Hallucination Severity (Dataset 3, 300 questions):

[0096] As can be seen from Table 6: (1) Compiled_verification reduced the severe hallucination rate from 42.67% at the baseline to 18.33%, a decrease of 24 percentage points. In legal scenarios, a "severe" level means that the model gives an incorrect legal judgment with a high confidence level of ≥0.8. If it is accepted directly without testing, it will directly lead to errors in the application of law. Therefore, the significant reduction in the severe hallucination rate is the core practical value of this invention. (2) The proportion of non-hallucinations (normal + mild) increased from 50.00% in the baseline to 61.67% in compiled_verification, an increase of about 12 percentage points. At the same time, hallucinations changed from being mainly "severe" (severe accounted for 42.67% in the baseline, and moderate accounted for only 7.33%) to being mainly "moderate" (severe decreased to 18.33% in compiled_verification, and moderate increased to 20.00%), indicating that the compiled verification "downgraded" a large number of incorrect answers that were originally output with high credibility to low credibility output, making the errors easier to be identified by the downstream manual review process; (3) There were 9.33% minor anomalies in compiled_verification (baseline was 0%). These minor anomalies were all correct answers but non-critical deviations were detected in certain dimensions of the compilation verification (such as boundary conditions discovered by truth propagation in the inference chain). They belong to "excessive caution on the correct answer", do not affect the quality of the final answer, and are not included in the illusion statistics.

[0097] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for eliminating illusions in the legal field based on a large language model of uncertainty theory, characterized in that: The method includes the following steps: Legal task object reception and preprocessing: Receive legal clauses, evidence texts, or statements of legal-related facts and events as task objects; parse the text content, candidate conclusions, and verification instructions; construct a structured representation of the legal task object; and retrieve relevant legal provisions from the legal knowledge base as contextual enhancement for the search. Initial reasoning and answer generation of the large language model: Structured legal questions and enhanced retrieval context are injected into the prompt template, and the large language model is invoked to generate an initial answer that includes option selection, reasoning process, model self-assessment confidence, and cited legal provisions; Legal logic compilation: The legal logic compiler, assisted by a large language model, is invoked to compile the natural language reasoning of the initial answer into an intermediate legal logic representation. The intermediate representation includes a set of atomic legal propositions, a set of legal rules, a sequence of reasoning chain steps, a list of cited legal provisions, and an option-proposition mapping relationship. Multi-channel formal verification: The intermediate representation of the legal logic is verified through at least two of the following three independent channels, and the at least two channels must include a response set procedural symbolic reasoning consistency verification channel: The first channel is for verifying legal provisions from the legal knowledge base; The second channel is for verifying the consistency of symbolic reasoning in the answer set procedure. The third channel is truth propagation verification based on the inference chain of uncertainty theory; Uncertainty theory reliability calibration and fusion: The reliability of each verification channel is regarded as the uncertainty measure of independent uncertain events, and the multi-channel uncertainty theory reliability is fused into calibration reliability based on the uncertainty measure product axiom; Risk assessment and hallucination determination output: Based on the calibration reliability, the consistency results of the answer set procedure and the reasonableness of the inference chain assignment, a comprehensive risk score is calculated. When the comprehensive risk score exceeds the preset threshold, it is determined that there is hallucination and the answer abandonment mechanism is triggered; otherwise, the verified final answer is output.

2. The method according to claim 1, characterized in that, The specific steps for receiving and preprocessing legal task objects include: The legal task object is structured and analyzed to extract the unique task identifier, the text to be verified, the candidate conclusions, the legal jurisdiction, the legal field classification, the time sensitivity indicator and the list of legal provisions, and to construct the legal task data items; Based on the question stem and option text, the legal citation regular pattern matching is used to identify the legal names and article numbers involved; The full text of the matching legal provisions is retrieved sequentially through the cascading architecture of the structured legal database adapter and the local rule base adapter, and used as the context for enhancing the retrieval to inject subsequent prompt templates.

3. The method according to claim 1, characterized in that, The specific steps involved in compiling the legal logic include: Construct legal formal logic compilation prompts and inject the question stems, options, and initial answers of the large language model into the compilation prompt template; The compiler prompts are executed by calling the large language model for verification, and a JSON output containing a list of propositions, a list of rules, a chain of inferences, cited legal provisions, and option-proposition mappings is obtained. The compilation output is parsed and semantically verified, including: checking whether the premise identifier of each step in the reasoning chain exists in the proposition set or rule set, checking whether the conclusion identifier exists in the proposition set, correcting the truth value range to the [0,1] interval, and recording all compilation errors.

4. The method according to claim 3, characterized in that, The atomic legal proposition comprises a unique proposition identifier, proposition natural language text, predicate, subject, truth value, source and negation flag; The legal rule comprises a unique rule identifier, legal basis name, rule text, antecedent proposition identifier list, consequent proposition identifier, exception case list and legal field label; The inference chain step comprises a step number, inference type, premise identifier list, conclusion identifier and the truth value of the conclusion of this step.

5. The method according to claim 1, characterized in that, The step that the first channel is used for legal knowledge base legal provision verification specifically comprises: Traversing the cited legal provision list in the legal logic intermediate representation and the legal basis names in the rule set, and verifying each legal provision reference sequentially through a cascaded knowledge base adapter; The cascaded knowledge base comprises: A structured legal database adapter and a local rule base adapter; Three-valued logic is adopted when calculating the knowledge base verification reliability: A verification reliability value is taken when the legal provision exists and has a high matching degree; A penalty value not lower than 0.5 is taken when the legal provision exists but the content does not match; A neutral value of 0.5 is taken when the legal provision is not found.

6. The method according to claim 1, characterized in that, The step that the second channel is used for answer set program symbolic inference consistency verification specifically comprises: Translating the legal logic intermediate representation into an answer set program, where the translation rules comprise: Translating the atomic legal proposition into a fact predicate or a negated fact predicate; Translating the legal rule into an implication rule; Translating the exception case into an integrity constraint; Translating the inference chain into a derivation rule; Translating the option-proposition mapping into an option support indication predicate; Invoking an answer set solver to solve the translated answer set program, enumerating all answer sets, and detecting logical inconsistency and integrity constraint violation; Extracting the supported option list from the solving result, and performing consistency comparison with the options predicted by the large language model.

7. The method according to claim 1, characterized in that, The step that the third channel is used for inference chain truth value propagation verification based on uncertainty theory specifically comprises: Constructing the premise and conclusion of each step in the inference chain of the legal logic intermediate representation into proposition nodes; Applying the inference rules of uncertain propositional logic to perform truth value propagation for each step in the inference chain according to its inference type: When the inference type is modus ponens, given the premise truth value T(A)=a and the implication truth value T(A→B)=b: If a+b>1, then T(B)=b, and the inference assignment is reasonable; T(A)=a and the implication truth value T(A→B)=b: If a+b>1, then T(B)=b, and the inference assignment is reasonable; If a+b=1, then T(B)=min(0.5, b), and the inference assignment is reasonable; If a+b<1, the inference assignment is pathological; When the inference type is modus tollens, given T(A→B)=a and T(B)=b: If a>b, then T(A)=1-a, and the inference assignment is reasonable; If a=b, then T(A)=max(1-a, 0.5), and the inference assignment is reasonable; If a<b, the inference assignment is pathological; When the inference type is hypothetical syllogism, given T(A→B)=a and T(B→C)=b: If a>0.5 and b>0.5, then T(A→C)=min(a, b), and the inference assignment is reasonable; If a+b≥1 and min(a, b)≤0.5, then T(A→C)=0.5, and the inference assignment is reasonable; If a+b<1, then the inference assignment is ill-conditioned. The results of the inference chain propagation are summarized into the inference chain truth propagation confidence: If any step has an ill-conditioned assignment, the confidence level is 0; otherwise, the minimum of the truth values ​​of all step conclusions is taken.

8. The method according to claim 1, characterized in that, The specific steps of uncertainty theory reliability calibration and fusion include: The reliability values ​​of each verification channel used were collected: truth propagation reliability of the inference chain, knowledge base verification reliability, and procedural consistency reliability of the answer set. The procedural consistency reliability of the answer set was 0.85 when consistent and 0.4 when inconsistent. If there are compilation errors in the intermediate representation of legal logic, a compilation quality factor is introduced, with a value of max(0.4, 1.0 - 0.05 × number of compilation errors); The collected reliability factors are treated as uncertainty measures of independent uncertain events, and the uncertainty measure product axiom is applied for fusion, i.e., the calibration reliability equals the minimum value of all factors: in, Propagate the confidence level to the truth value of the inference chain. To verify the reliability of the knowledge base To answer the question of program consistency reliability, This is the compilation quality factor.

9. The method according to claim 1, characterized in that, The steps for risk assessment and hallucination determination output specifically include: The initial risk score is 0, and the following risk signals are added sequentially: If the calibration reliability is lower than the dropout threshold, the risk score increases by the difference between the dropout threshold and the calibration reliability. If the response set program detects a logical inconsistency, the risk score is increased by the first weight; If there are ill-conditioning steps in the inference chain, the risk score is increased by a second weight for each ill-conditioning step; When the risk score exceeds the judgment threshold, it is determined that there is a hallucination, triggering the abandonment mechanism, marking the final answer as abandoned, and setting the credibility to 0; otherwise, the verified initial answer is output, and the credibility is taken as the calibrated reliability value. The rejection threshold, the first weight, the second weight, and the judgment threshold are preset positive numbers.

10. The method according to claim 1, characterized in that, The initial reasoning and answer generation steps of the large language model also include a multi-sample voting reliability calibration sub-method based on uncertainty theory, specifically including: Based on diverse information inputs, prompts and variations, or sampling configurations, multiple independent samples are taken from the same legal clause, evidence text, or legal-related factual event statement task object. Each sampling acquires the option selection, model self-assessment confidence, and inference process. Construct a belief distribution profile, and calculate the exact set lower bound, single-option lower bound, and complementary event lower bound for each option. The exact set lower bound is the minimum confidence among all support samples for that option, and the complementary event lower bound is the minimum confidence among all samples excluding that option. Detecting contradictory options: If the self-assessed confidence levels of two or more different options all exceed the contradictory reliability threshold, then a contradiction is determined to exist; High-confidence samples are selected according to the retention ratio and weighted voting is performed. The fusion confidence of the winning option is the minimum value of the minimum confidence of its supporting samples, the consensus ratio, and the lower bound of the prediction set. If a contradiction is detected or the fusion confidence is lower than the rejection confidence threshold, this sub-method outputs a rejection flag.

Images