Large language model illusion suppression method and system based on formalized proof

Abstract

The invention relates to the field of artificial intelligence, and discloses a large language model illusion suppression method and system based on formalized proof, and the method comprises the steps: carrying out the logic mapping of the input and output of a language model through a formalized logic verification framework, and carrying out the logic verification of the output of the language model according to rule matching and recursive reasoning, identifying model illusion data; on the basis of an association rule mining algorithm, performing illusion mode mining on the model illusion data; clustering the illusion modes by utilizing clustering analysis to obtain a clustering result of the illusion modes; and based on a clustering result of the illusion mode, using a rule learning algorithm to formulate an inhibition rule, generating a branch rule of the decision tree, and performing illusion inhibition on an output result of the language model according to the inhibition rule and the branch rule. Through automatic rule learning and association rule mining, the occurrence of illusion modes is reduced, and the reasoning quality of the language model is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a large language model hallucination suppression method and system based on formal proof. Background Art

[0002] Large language model hallucination suppression uses specific techniques and methods to reduce or eliminate hallucinations that occur during language model inference. During the inference process of a large language model, the model may generate some unrealistic, irrelevant, or inaccurate content, which is called "hallucinations." Hallucinations can be caused by data bias, overfitting during training, inconsistent inference paths, and other reasons. Hallucination suppression aims to improve the model's inference process, calibrate output results, and enhance model stability and credibility.

[0003] Formal proof is a method based on formal logic verification and proof to reduce and eliminate hallucinations in language models. This method involves formally modeling the reasoning process and model output, and using logical reasoning rules and mathematical proofs to ensure the correctness of the reasoning process.

[0004] Specifically, existing large language models (such as the GPT series and LLaMA) generate text through massive data training, but their output often has the following problems:

[0005] 1. Factual errors: Generating content that conflicts with real-world knowledge (e.g., fictional historical events).

[0006] 2. Logical contradiction: contradictory conclusions appear in contextual reasoning.

[0007] 3. Semantic ambiguity: Deviations in understanding user questions lead to irrelevant answers.

[0008] At the same time, existing large language model hallucination suppression methods still have the following shortcomings:

[0009] 1. Existing solutions mostly rely on post-processing filtering or probability threshold adjustment, but lack formal verification of input-output logical consistency, making it difficult to eradicate deep-seated hallucination problems.

[0010] 2. How to make good use of the hallucination data of the large language model and identify effective large language model hallucination patterns.

[0011] 3. How to derive suppression rules for large language model hallucinations and effectively embed these rules into the inference process.

[0012] Currently, no effective solutions have been proposed for the problems in related technologies. Summary of the Invention

[0013] In view of this, the purpose of this application is to provide a large language model hallucination suppression method and system based on formal proof to overcome the above-mentioned technical problems existing in the existing related technologies.

[0014] To this end, the specific technical solutions adopted in the present invention are as follows:

[0015] According to one aspect of the present invention, a large language model hallucination suppression method based on formal proof is provided, comprising:

[0016] Using a formal logic verification framework, the input and output of the language model are logically mapped. Based on rule matching and recursive reasoning, the output of the language model is logically verified to identify model hallucination data.

[0017] Based on the association rule mining algorithm, the hallucination pattern of the model hallucination data is mined; the hallucination pattern is clustered using cluster analysis to obtain the clustering results of the hallucination pattern;

[0018] Based on the clustering results of hallucination patterns, a rule learning algorithm is used to formulate suppression rules and generate branching rules of the decision tree. The output results of the language model are subjected to hallucination suppression according to the suppression rules and branching rules.

[0019] Furthermore, a formal logic verification framework is used to logically map the input and output of the language model. Based on rule matching and recursive reasoning, the output of the language model is logically verified to identify model hallucination data, including:

[0020] Parse user input questions into logical propositions and clarify the entities, relationships and constraints in the user input questions;

[0021] Convert the language model output into a verifiable logical structure and construct a joint logical space with the user input question;

[0022] Use logic programming language to build a domain knowledge rule base, and verify the logical consistency of user input questions and output content through rule matching and recursive reasoning verification in the domain knowledge rule base;

[0023] When a logical consistency problem is detected, model regeneration or risk warning is triggered, and the input and output data of the language model at this time are collected to form model hallucination data.

[0024] Furthermore, using logic programming language to build a domain knowledge rule base includes:

[0025] Collect relevant knowledge from domain experts and data sources, extract knowledge points and establish a knowledge system;

[0026] Based on domain knowledge, establish a conceptual model of the relevant field and determine the relationship between various concepts;

[0027] Use logic programming language to define domain rules, define the premises and conclusions in the reasoning process, and ensure logical consistency between rules;

[0028] The defined domain rules and known facts are stored in the knowledge base and integrated with the reasoning engine to obtain the domain knowledge rule base.

[0029] Furthermore, based on the association rule mining algorithm, the hallucination pattern mining of the model hallucination data includes:

[0030] Converting the model hallucination data into a transaction dataset, and making the model hallucination data suitable for an association rule mining algorithm by representing the features in each inference as items in the transaction dataset;

[0031] Determine the types of hallucination patterns and hallucination characteristics that need to be mined, and clarify the goals of the mining task, which includes obtaining a certain type of hallucination pattern;

[0032] Determine the association rule mining algorithm and configure the parameters of the association rule mining algorithm, including minimum support and minimum confidence;

[0033] By using the connection, pruning and screening techniques in the association rule mining algorithm, we can mine the transaction data set for association rules, find frequent item sets, and generate several preliminary association rules.

[0034] By calculating the support and confidence of the preliminary association rules and screening the rules based on the support and confidence, the association rules to be analyzed are screened out from the preliminary association rules;

[0035] According to the selected association rules to be analyzed, hallucination patterns related to the language model hallucination are determined.

[0036] Furthermore, we use the connection, pruning and screening techniques in the association rule mining algorithm to mine association rules on the transaction data set, find frequent item sets, and generate several preliminary association rules including:

[0037] The number of occurrences of each item in the transaction database is counted, and if the support of the item set is greater than or equal to the minimum support, the corresponding item set is added to the frequent item set;

[0038] Based on frequent item sets, a candidate k-item set set is generated through the connection technology in the association rule mining algorithm;

[0039] Determine whether the subset of each candidate k-item set belongs to the frequent k-1 item set set. If not, delete the corresponding candidate k-item set. If so, retain the corresponding candidate k-item set.

[0040] For the pruned candidate k-itemset set, the support estimate of each candidate k-itemset is calculated by the prior probability. If the support estimate is greater than the preset value, the corresponding candidate k-itemset is added to the frequent k-itemset set; otherwise, the corresponding candidate k-itemset is removed.

[0041] The support of the selected candidate k-item sets is counted, and the candidate k-item sets with support greater than or equal to the minimum support are taken as the final selected frequent itemsets and converted into preliminary association rules.

[0042] Furthermore, based on the selected association rules to be analyzed, it is determined that the hallucination patterns related to the language model hallucination include:

[0043] According to the premise and conclusion of the association rule to be analyzed, the type of reasoning error in the association rule to be analyzed is determined, and according to the type of reasoning error, the relevant hallucination pattern features are marked;

[0044] Obtain the cause of each hallucination pattern and deduce the causal relationship of the reasoning error type based on the premise and conclusion of the association rule to be analyzed; organize the causal relationship and construct the causal relationship of the hallucination pattern;

[0045] According to the causal relationship of hallucination patterns and through logical reasoning, the logical relationship between different hallucination patterns is analyzed; based on the logical connection, the hallucination pattern related to the language model hallucination is finally determined.

[0046] Furthermore, cluster analysis is used to cluster the hallucination patterns, and the clustering results of the hallucination patterns include:

[0047] Apply clustering algorithms to perform cluster analysis on hallucination patterns and select distance metrics;

[0048] According to the hallucination pattern characteristics and by calculating the similarity between each hallucination pattern, the hallucination patterns are grouped to obtain several hallucination pattern types.

[0049] Furthermore, based on the clustering results of the hallucination patterns, a rule learning algorithm is used to formulate suppression rules and generate branching rules of the decision tree. The output results of the language model are subjected to hallucination suppression according to the suppression rules and branching rules, including:

[0050] Sort out the common features of each hallucination pattern type, combine the hallucination problems of each hallucination pattern type, and determine the suppression needs of each hallucination pattern type;

[0051] Using rule learning algorithms, we derive inhibition rules from the common features of hallucination patterns and the need for inhibition, identify the causal relationships of hallucination patterns, and logically process the inhibition rules to correct errors in language model reasoning.

[0052] The common features of hallucination patterns are used as nodes of the decision tree, and each node is assigned a corresponding inhibition requirement; based on the characteristics of each node, branching rules are generated;

[0053] The generated inhibition rules and branching rules are embedded in the reasoning process of the language model, and the incidence of hallucination patterns in the language model reasoning before and after inhibition is compared to evaluate the effectiveness of the inhibition rules.

[0054] Furthermore, rule learning algorithms are used to derive inhibition rules from the common features of hallucination patterns and the need for inhibition. The causal relationships of hallucination patterns are identified, and the inhibition rules are logically processed to correct errors in language model reasoning, including:

[0055] Correlating the common features of each hallucination pattern type with the suppression demand, obtaining a list containing each hallucination pattern type and its corresponding suppression demand;

[0056] Inductive logic programming is used to derive inhibition rules from the common features of the organized hallucination patterns and inhibition requirements, and the inhibition rules are made logical by learning the causal relationship of the common features.

[0057] According to another aspect of the present invention, a large language model hallucination suppression system based on formal proof is also provided, including a model hallucination data acquisition module, a hallucination pattern mining module and a model output correction module.

[0058] The model hallucination data acquisition module is used to use a formal logic verification framework to logically map the input and output of the language model, and to logically verify the output of the language model based on rule matching and recursive reasoning to identify the model hallucination data; the hallucination pattern mining module is used to mine hallucination patterns of the model hallucination data based on the association rule mining algorithm; the hallucination patterns are clustered using cluster analysis to obtain the clustering results of the hallucination patterns; the model output correction module is used to formulate suppression rules based on the clustering results of the hallucination patterns using a rule learning algorithm, and to generate branching rules of the decision tree, and to suppress the hallucinations of the output results of the language model according to the suppression rules and branching rules.

[0059] The present invention includes the following beneficial effects:

[0060] This invention embeds logic verification during the generation phase, reducing error rates by over 40% compared to traditional post-processing approaches (based on medical QA benchmarks). It provides specific reasons for logic verification failures (e.g., "output violates causal rule R123"), enhancing user trust. By replacing the rule base, it can adapt to different scenarios (e.g., legal contract review, scientific paper fact checking).

[0061] 2. By applying association rule mining algorithms to process the hallucination data of the language model, it is possible to identify and classify different types of hallucination patterns, and through cluster analysis, different hallucination patterns can be mined from a large amount of inference data.

[0062] 3. By mapping the common characteristics of hallucination patterns and the need for suppression into a decision tree and using a rule-learning algorithm to generate suppression rules, we can achieve automated error correction during language model inference. This approach automatically adjusts the inference process based on the characteristics of hallucination patterns and the need for suppression, reducing the occurrence of hallucination patterns and improving the reliability of inference results. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0064] Figure 1 is a flowchart of a large language model hallucination suppression method based on formal proof according to an embodiment of the present invention;

[0065] Figure 2 2. It is a principle block diagram of a large language model hallucination suppression system based on formal proof according to an embodiment of the present invention;

[0066] In the picture:

[0067] 1. Model hallucination data acquisition module; 2. Hallucination pattern mining module; 3. Model output correction module. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0069] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0070] According to an embodiment of the present invention, a method and system for suppressing hallucinations in large language models based on formal proofs are provided. A formal proof-driven hallucination suppression framework is proposed. This framework proactively prevents and corrects hallucinations in large models by mapping natural language questions and outputs into logical propositions and verifying their logical consistency using formal methods (such as Prolog logic programming).

[0071] The present invention will now be further described with reference to the accompanying drawings and specific embodiments. Figure 1 As shown, according to one embodiment of the present invention, a large language model hallucination suppression method based on formal proof is provided, including:

[0072] S1. Use a formal logic verification framework to logically map the input and output of the language model. Based on rule matching and recursive reasoning, perform logical verification on the output of the language model to identify model hallucination data.

[0073] In one embodiment, a formal logic verification framework is used to perform a logical mapping between the input and output of the language model, and the output of the language model is logically verified based on rule matching and recursive reasoning. Identifying model hallucination data includes:

[0074] Parse user input questions into logical propositions, identifying the entities, relationships, and constraints within them. Transform the language model output into a verifiable logical structure and construct a joint logical space with the user input question. Utilize a logic programming language to construct a domain knowledge rule base, and verify the logical consistency of the user input question and output content through rule matching and recursive reasoning within the domain knowledge rule base. When logical consistency issues are detected, model regeneration or risk warnings are triggered, and the language model input and output data at that time are collected to form model hallucination data. This includes input-output logical mapping, a formal proof verification engine, and a dynamic feedback correction mechanism. Recursive reasoning verification of the rule base is implemented using logic programming languages ​​such as Prolog.

[0075] In one embodiment, constructing a domain knowledge rule base using a logic programming language includes:

[0076] Collect relevant knowledge from domain experts and data sources, extract knowledge points and establish a knowledge system; based on domain knowledge, establish a conceptual model of the relevant domain and determine the relationship between each concept; use logic programming language to define domain rules, define the premises and conclusions in the reasoning process, and ensure logical consistency between rules; store the defined domain rules and known facts (domain data) in the knowledge base, and integrate the reasoning engine to obtain a domain knowledge rule base.

[0077] Parse user input questions into first-order logic propositions (such as predicate logic expressions) to clarify the entities, relationships, and constraints in the problem. Convert the output content of the large model into a verifiable logical structure (such as the "if A then B" implication), and construct a joint logical space with the input proposition. Based on logical programming languages ​​​​such as Prolog, build a domain knowledge rule base (such as medical causal chains, legal text logical dependencies), and verify whether the output violates the preset logical constraints through rule matching and recursive reasoning. Support dynamic expansion of the rule base to adapt to different vertical fields (such as financial compliance checks). If a logical contradiction or factual conflict is detected, trigger model regeneration or mark a risk warning (such as "the current conclusion does not pass the logical verification of legal text X").

[0078] S2. Based on the association rule mining algorithm, the hallucination patterns of the model hallucination data are mined; the hallucination patterns are clustered using cluster analysis to obtain the clustering results of the hallucination patterns.

[0079] In one embodiment, mining hallucination patterns from model hallucination data based on an association rule mining algorithm includes:

[0080] The model hallucination data is converted into a transaction data set, and the features in each reasoning (such as reasoning path, information omission, and wrong choice) are represented as items in the transaction data set to make the model hallucination data suitable for the association rule mining algorithm; the types of hallucination patterns to be mined (such as reasoning errors, information omissions, and missing background knowledge) and hallucination features are determined, and the goals for the mining task are clarified, and the goals include obtaining a certain type of hallucination pattern; the association rule mining algorithm is determined, and the parameters of the association rule mining algorithm are configured, including minimum support and minimum confidence; the connection, pruning and screening techniques in the association rule mining algorithm are used to mine association rules on the transaction data set, find frequent item sets, and generate several preliminary association rules; by calculating the support (the frequency of item sets appearing in the data) and confidence (the reliability of the rules) of the preliminary association rules, and screening rules based on the support and confidence, the association rules to be analyzed are screened out from the preliminary association rules; based on the screened association rules to be analyzed, the hallucination patterns related to the language model hallucination are determined.

[0081] In one embodiment, association rule mining is performed on the transaction data set using the connection, pruning, and screening techniques in the association rule mining algorithm to find frequent item sets and generate several preliminary association rules, including:

[0082] The number of occurrences of each item in the transaction database is counted, and if the support of the item set is greater than or equal to the minimum support, the corresponding item set is added to the frequent item set; based on the frequent item set, and through the connection technology in the association rule mining algorithm, a candidate k-item set set is generated; it is determined whether the subset of each candidate k-item set belongs to the frequent k-1 item set set, if not, the corresponding candidate k-item set is deleted, if so, the corresponding candidate k-item set is retained; for the pruned candidate k-item set, the support estimate of each candidate k-item set is calculated by the prior probability, if the support estimate is greater than the preset value, the corresponding candidate k-item set is added to the frequent k-item set set, otherwise, the corresponding candidate k-item set is removed; the support of the screened candidate k-item sets is counted, and the candidate k-item sets with support greater than or equal to the minimum support are used as the final selected frequent item sets and converted into preliminary association rules.

[0083] Among them, the prior probability calculation formula is:

[0084]

[0085] Where P(c) represents the prior probability of candidate k-itemset c, that is, the estimated support; P(x) represents the support of a single-element subset x in the candidate k-itemset, which can be understood as the probability that a specific error type (such as a specific logical error in the reasoning path) occurs in all transactions; P(cx) represents the support of the remaining part after removing element x from the candidate k-itemset c, reflecting the frequency of occurrence of the remaining error combination when a specific type of error does not exist; n represents the number of items contained in the candidate k-itemset c. In association rule mining, it refers to the number of errors or features in a specific candidate k-itemset; k represents the number of items in the item set, that is, the number of errors or features contained in a specific candidate item set.

[0086] In one embodiment, determining a hallucination pattern related to the language model hallucination based on the selected association rules to be analyzed includes:

[0087] According to the premise and conclusion of the association rule to be analyzed, determine the type of reasoning error in the association rule to be analyzed, and mark the relevant hallucination pattern characteristics according to the type of reasoning error. For example, if the reasoning error is caused by missing information, it is marked as "information omission"; obtain the cause of each hallucination pattern, and deduce the causal relationship of the reasoning error type according to the premise and conclusion of the association rule to be analyzed. For example, if the model omits certain background knowledge in the reasoning process, resulting in an incorrect conclusion, this is a manifestation of causal relationship; organize the causal relationship and construct the causal relationship of the hallucination pattern; according to the causal relationship of the hallucination pattern and through logical reasoning, analyze the logical relationship between different hallucination patterns; based on the logical connection, finally determine the hallucination pattern related to the language model hallucination.

[0088] During the inference process of large language models, some erroneous or inaccurate outputs, known as "hallucinations," may occur. These hallucinations are typically caused by incorrect reasoning paths, missing information, or a lack of background knowledge. The goal of hallucination suppression is to reduce or eliminate these erroneous outputs through technical means, ensuring more accurate and reliable model reasoning. Association rule mining is used to discover underlying patterns in data, specifically hallucination patterns that occur during model reasoning. Hallucination patterns refer to recurring error types or patterns of reasoning errors in model output, such as incorrect reasoning paths, missing information, or missing background knowledge.

[0089] To adapt the association rule mining algorithm for hallucination pattern identification, the language model's inference data is converted into a transaction dataset. In this transaction dataset, each "transaction" represents the characteristics of an inference process, including elements such as the inference path and incorrect choices. The type of hallucination pattern is identified. Examples include reasoning errors, information omissions, and missing background knowledge. This hallucination pattern type serves as the target of the subsequent mining process, allowing the association rule mining algorithm to accurately identify different types of hallucinations. Association rule mining involves identifying frequent item sets, generating preliminary association rules, and screening these rules. After generating candidate rules, the support and confidence of each rule are calculated. The selected association rules are then used to further analyze the causal relationships within the hallucination patterns. For example, if the model omits background knowledge during inference, resulting in an inference error, the root cause of the error can be deduced from the causal relationships within the rules. Causal relationships help understand the connections between different hallucination patterns and provide a basis for the subsequent formulation of corrective rules.

[0090] In one embodiment, clustering the hallucination patterns is performed using cluster analysis, and the clustering results of the hallucination patterns include:

[0091] Clustering algorithm is applied to cluster analysis of hallucination patterns, and distance metric is selected. Based on hallucination pattern characteristics and by calculating the similarity between hallucination patterns, hallucination patterns are grouped to obtain several hallucination pattern types.

[0092] Using a clustering analysis algorithm, hallucination patterns are grouped into different categories based on their similarities. For example, if some hallucination patterns are caused by errors in reasoning paths, these patterns will be clustered together; while other hallucination patterns are caused by missing information or lack of background knowledge, they will be clustered into another category. When applying the clustering algorithm, a distance metric is selected. For example, Euclidean distance or cosine similarity can be used to measure the similarity between two hallucination patterns.

[0093] S3. Based on the clustering results of the hallucination pattern, a rule learning algorithm is used to formulate suppression rules and generate branching rules of the decision tree. The output results of the language model are subjected to hallucination suppression according to the suppression rules and branching rules.

[0094] In one embodiment, based on the clustering results of the hallucination patterns, a suppression rule is formulated using a rule learning algorithm, and a branching rule of a decision tree is generated. Performing hallucination suppression on the output results of the language model according to the suppression rule and the branching rule includes:

[0095] The common features of each hallucination pattern type are sorted out, and the inhibition requirements of each hallucination pattern type are determined in combination with the hallucination problems of each hallucination pattern type; the inhibition rules are derived from the common features and inhibition requirements of the hallucination patterns using rule learning algorithms, and the causal relationship of the hallucination patterns is identified, and the inhibition rules are logically processed to correct errors in language model reasoning; the common features of the hallucination patterns are used as nodes of the decision tree, and corresponding inhibition requirements are assigned to each node; branching rules are generated based on the characteristics of each node; the generated inhibition rules and branching rules are embedded in the reasoning process of the language model, and the incidence of hallucination patterns in the language model reasoning before and after inhibition is compared to evaluate the effectiveness of the inhibition rules.

[0096] Some hallucination patterns are caused by reasoning errors, while others result from information omissions or a lack of background knowledge. Therefore, by analyzing the common characteristics of these hallucination patterns, we can clarify the need to suppress each hallucination pattern. For example, reasoning errors require correction through enhanced verification steps or review of reasoning paths. Information omissions require supplementation with more background knowledge or enhanced understanding of the context during reasoning.

[0097] Rule-learning algorithms derive suppression rules from the common characteristics of hallucination patterns and suppression requirements, concretizing them into actions to be taken during model inference. Rule-learning algorithms automatically discover potential suppression rules by learning patterns and relationships from large amounts of data and logically derive them. Specifically, through methods such as inductive logic programming, complex suppression rules are derived to improve the accuracy of corrections.

[0098] The common features of hallucination patterns are used as nodes in a decision tree, with each node corresponding to a specific hallucination pattern type. Based on the characteristics of each node, corresponding branching rules are generated. These branching rules define the correction methods for different scenarios. For example, if the model makes a "path error" during reasoning, a branch of the decision tree will instruct the model to add path verification. If the error is caused by "information omission", the branching rule will instruct the model to supplement relevant background knowledge.

[0099] To ensure the effectiveness of the suppression rule, a comparison is usually performed before and after the suppression to evaluate the incidence of hallucination patterns in model reasoning. By evaluating the changes before and after the suppression, the effectiveness of the suppression rule can be judged and the suppression strategy can be further optimized.

[0100] In one embodiment, a rule learning algorithm is used to derive suppression rules from the common features of hallucination patterns and suppression requirements, identify the causal relationship of hallucination patterns, and logically process the suppression rules to correct errors in language model reasoning, including:

[0101] The common features of each hallucination pattern type are associated with the inhibition requirements to obtain a list containing each hallucination pattern type and its corresponding inhibition requirements; inductive logic programming is used to derive inhibition rules from the sorted common features of hallucination patterns and inhibition requirements, and the inhibition rules are made logical by learning the causal relationship of the common features.

[0102] Among them, inductive logic programming is a method that combines machine learning and logic programming. It generates rules through reasoning and induction from a set of known positive and negative examples. The goal of inductive logic programming is to learn logical rules that can be applied to these examples from known examples, usually expressed in the form of logical formulas (for example, predicate logic formulas). Specifically, given a set of positive examples (correct examples) and negative examples (incorrect examples), the goal is to derive one or more rules through induction that can distinguish between positive and negative examples and explain why positive examples are true and negative examples are false. These rules are in the form of Horn clauses, that is, "if...then..."

[0103] like Figure 2 As shown, according to another embodiment of the present invention, a large language model hallucination suppression system based on formal proof is also provided, including a model hallucination data acquisition module 1, a hallucination pattern mining module 2 and a model output correction module 3.

[0104] The model hallucination data acquisition module 1 is used to use a formal logic verification framework to perform logical mapping on the input and output of the language model, and to perform logical verification on the output of the language model based on rule matching and recursive reasoning to identify model hallucination data.

[0105] The hallucination pattern mining module 2 is used to mine hallucination patterns from the model hallucination data based on the association rule mining algorithm; cluster the hallucination patterns using cluster analysis to obtain clustering results of the hallucination patterns.

[0106] The model output correction module 3 is used to formulate suppression rules based on the clustering results of the hallucination pattern using a rule learning algorithm and generate branching rules of a decision tree, and to suppress hallucinations on the output results of the language model according to the suppression rules and branching rules.

[0107] In order to facilitate understanding of the above technical solutions of the present invention, the working principle of the present invention in actual process is described in detail below.

[0108] The present invention relates to a technical method and system for suppressing "hallucinations" (i.e., outputs that are inconsistent with facts or logic) in content generated by a large language model (LLM) through formal logic verification. The method is particularly suitable for scenarios such as medicine, law, and science that have high requirements for factual accuracy.

[0109] Define domain-related predicates, such as `causes(Disease,Symptom)` to indicate the causal relationship between disease and symptoms) and axioms (such as

[0110] Example Prolog snippet:

[0111] ```prolog% Medical domain rules: drug contraindication verification.

[0112] contraindication(Drug,Condition):-

[0113] patient_condition(Condition),

[0114] drug_interaction(Drug,Condition).

[0115] % Input proposition: The patient has high blood pressure, and the model recommends taking drug X.

[0116] % Verification logic: If drug X is contraindicated with hypertension, a warning is triggered.

[0117] validate_output:-

[0118] patient_condition(hypertension),

[0119] suggested_drug(X),

[0120] (contraindication(X,hypertension)→warn('Contraindicated drug risk')).

[0121] ```

[0122] Using dependency parsing and semantic role labeling, natural language is converted into logical expressions, such as "Penicillin cannot be used for asthma patients" → `contraindication(penicillin,asthma)`. The user asks a question, the large model generates candidate answers, the logical parser extracts the proposition, and the Prolog engine verifies the logical consistency. If the answer passes, it is output; otherwise, a correction is triggered.

[0123] In addition, the process of suppressing the large model illusion includes the following.

[0124] Inference transactions during model inference are recorded, and each inference transaction is represented as a transaction dataset containing features. Each feature is mapped to an item in the transaction. For example, a step in the inference path can be represented as "Path Error 1," missing information can be represented as "Information Missing 1," and so on.

[0125] The reasoning transactions are converted into transaction datasets, with each transaction containing multiple items. The goal is to mine hallucination patterns such as reasoning error patterns, information omission patterns, and background knowledge loss patterns. Minimum support and minimum confidence settings: Set the minimum support to 0.1 (i.e., a hallucination pattern appears in at least 10% of transactions) and the minimum confidence to 0.8 (i.e., the credibility of the rule is above 80%). Generate frequent item sets using connection, pruning, and screening techniques. Generate preliminary association rules from the frequent item sets, and filter out the following rules by calculating support and confidence. Based on these rules, further analyze and identify hallucination patterns related to language model hallucinations.

[0126] Based on the association rule analysis results, cluster analysis was performed to group different types of hallucination patterns. Euclidean distance was used to measure the similarity between hallucination patterns. Using the clustering algorithm, the following hallucination patterns were divided into several categories.

[0127] Based on the clustering results, a rule learning algorithm is used to derive suppression rules. Based on the common features of each hallucination pattern, the suppression requirement is determined. Inductive logic programming is used to learn the suppression rules from positive and negative examples. Based on the suppression rules, a decision tree is constructed and branching rules are generated. The common features of each hallucination pattern type are used as nodes in the decision tree, and the corresponding suppression requirements are assigned. By embedding these suppression and branching rules in the model inference process, comparative experiments are conducted to evaluate the changes in the incidence of hallucination patterns before and after suppression.

[0128] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A large language model hallucination suppression method based on formal proof, characterized by: include: Using a formal logic verification framework, the input and output of the language model are logically mapped. Based on rule matching and recursive reasoning, the output of the language model is logically verified to identify model hallucination data. Based on the association rule mining algorithm, the hallucination pattern of the model hallucination data is mined; Cluster analysis is used to cluster hallucination patterns and obtain clustering results of hallucination patterns; Based on the clustering results of hallucination patterns, a rule learning algorithm is used to formulate suppression rules and generate branching rules of the decision tree. The output results of the language model are subjected to hallucination suppression according to the suppression rules and branching rules.

2. The large language model hallucination suppression method based on formal proof according to claim 1 is characterized in that: The formal logic verification framework is used to perform logical mapping between the input and output of the language model, and the output of the language model is logically verified based on rule matching and recursive reasoning. Identifying model hallucination data includes: Parse user input questions into logical propositions and clarify the entities, relationships and constraints in the user input questions; Convert the language model output into a verifiable logical structure and construct a joint logical space with the user input question; Use logic programming language to build a domain knowledge rule base, and verify the logical consistency of user input questions and output content through rule matching and recursive reasoning verification in the domain knowledge rule base; When a logical consistency problem is detected, model regeneration or risk warning is triggered, and the input and output data of the language model at this time are collected to form model hallucination data.

3. The large language model hallucination suppression method based on formal proof according to claim 2 is characterized in that: The use of a logic programming language to construct a domain knowledge rule base includes: Collect relevant knowledge from domain experts and data sources, extract knowledge points and establish a knowledge system; Based on domain knowledge, establish a conceptual model of the relevant field and determine the relationship between various concepts; Use logic programming language to define domain rules, define the premises and conclusions in the reasoning process, and ensure logical consistency between rules; The defined domain rules and known facts are stored in the knowledge base and integrated with the reasoning engine to obtain the domain knowledge rule base.

4. The large language model hallucination suppression method based on formal proof according to claim 1 is characterized in that The mining of hallucination patterns of model hallucination data based on the association rule mining algorithm includes: Converting the model hallucination data into a transaction dataset, and making the model hallucination data suitable for an association rule mining algorithm by representing the features in each inference as items in the transaction dataset; Determine the types of hallucination patterns and hallucination characteristics that need to be mined, and clarify the goals of the mining task, which includes obtaining a certain type of hallucination pattern; Determine the association rule mining algorithm and configure the parameters of the association rule mining algorithm, including minimum support and minimum confidence; By using the connection, pruning and screening techniques in the association rule mining algorithm, we can mine the transaction data set for association rules, find frequent item sets, and generate several preliminary association rules. By calculating the support and confidence of the preliminary association rules and screening the rules based on the support and confidence, the association rules to be analyzed are screened out from the preliminary association rules; According to the selected association rules to be analyzed, hallucination patterns related to the language model hallucination are determined.

5. The large language model hallucination suppression method based on formal proof according to claim 4 is characterized in that: The connection, pruning and screening techniques in the association rule mining algorithm are used to mine association rules on the transaction data set, find frequent item sets, and generate several preliminary association rules including: The number of occurrences of each item in the transaction database is counted, and if the support of the item set is greater than or equal to the minimum support, the corresponding item set is added to the frequent item set; Based on frequent item sets, a candidate k-item set set is generated through the connection technology in the association rule mining algorithm; Determine whether the subset of each candidate k-item set belongs to the frequent k-1 item set set. If not, delete the corresponding candidate k-item set. If so, retain the corresponding candidate k-item set. For the pruned candidate k-itemset set, the support estimate of each candidate k-itemset is calculated by the prior probability. If the support estimate is greater than the preset value, the corresponding candidate k-itemset is added to the frequent k-itemset set; otherwise, the corresponding candidate k-itemset is removed. The support of the selected candidate k-item sets is counted, and the candidate k-item sets with support greater than or equal to the minimum support are taken as the final selected frequent itemsets and converted into preliminary association rules.

6. The large language model hallucination suppression method based on formal proof according to claim 4 is characterized in that: Determining the hallucination pattern related to the language model hallucination based on the filtered association rules to be analyzed includes: According to the premise and conclusion of the association rule to be analyzed, the type of reasoning error in the association rule to be analyzed is determined, and according to the type of reasoning error, the relevant hallucination pattern features are marked; Obtain the cause of each hallucination pattern and deduce the causal relationship of the reasoning error type based on the premise and conclusion of the association rule to be analyzed; organize the causal relationship and construct the causal relationship of the hallucination pattern; According to the causal relationship of hallucination patterns and through logical reasoning, the logical relationship between different hallucination patterns is analyzed; based on the logical connection, the hallucination pattern related to the language model hallucination is finally determined.

7. The large language model hallucination suppression method based on formal proof according to claim 1 is characterized in that: The clustering of the hallucination patterns by cluster analysis to obtain the clustering results of the hallucination patterns includes: Apply clustering algorithms to perform cluster analysis on hallucination patterns and select distance metrics; According to the hallucination pattern characteristics and by calculating the similarity between each hallucination pattern, the hallucination patterns are grouped to obtain several hallucination pattern types.

8. The large language model hallucination suppression method based on formal proof according to claim 1 is characterized in that: The clustering result based on the hallucination pattern, using a rule learning algorithm to formulate a suppression rule, and generating a branching rule of a decision tree, and performing hallucination suppression on the output result of the language model according to the suppression rule and the branching rule includes: Sort out the common features of each hallucination pattern type, combine the hallucination problems of each hallucination pattern type, and determine the suppression needs of each hallucination pattern type; Using rule learning algorithms, we derive inhibition rules from the common features of hallucination patterns and the need for inhibition, identify the causal relationships of hallucination patterns, and logically process the inhibition rules to correct errors in language model reasoning. The common features of hallucination patterns are used as nodes of the decision tree, and each node is assigned a corresponding inhibition requirement; based on the characteristics of each node, branching rules are generated; The generated inhibition rules and branching rules are embedded in the reasoning process of the language model, and the incidence of hallucination patterns in the language model reasoning before and after inhibition is compared to evaluate the effectiveness of the inhibition rules.

9. The large language model hallucination suppression method based on formal proof according to claim 8 is characterized in that: The rule learning algorithm is used to derive suppression rules from the common features of hallucination patterns and suppression requirements, identify the causal relationship of hallucination patterns, and logically process the suppression rules to correct errors in language model reasoning, including: Correlating the common features of each hallucination pattern type with the suppression demand, obtaining a list containing each hallucination pattern type and its corresponding suppression demand; Inductive logic programming is used to derive inhibition rules from the common features of the organized hallucination patterns and inhibition requirements, and the inhibition rules are made logical by learning the causal relationship of the common features.

10. A large language model hallucination suppression system based on formal proof, used to implement the large language model hallucination suppression method based on formal proof according to any one of claims 1 to 9, characterized in that: It includes a model hallucination data acquisition module, a hallucination pattern mining module and a model output correction module; The model hallucination data acquisition module is used to use a formal logic verification framework to logically map the input and output of the language model, and to perform logical verification on the output of the language model based on rule matching and recursive reasoning to identify model hallucination data. The hallucination pattern mining module is used to mine hallucination patterns of model hallucination data based on association rule mining algorithm; Cluster analysis is used to cluster hallucination patterns and obtain clustering results of hallucination patterns; The model output correction module is used to formulate suppression rules based on the clustering results of the hallucination pattern using a rule learning algorithm and generate branching rules of the decision tree, and to suppress hallucinations on the output results of the language model according to the suppression rules and branching rules.

Images