Illusion detection method, device and equipment for large model and chip

Abstract

The application introduces a hallucination detection method for a large model, which comprises: obtaining an input question and an initial response text generated by a first large model for the question; generating a non-inductive query and an inductive query based on the question, wherein the non-inductive query is a query that does not contain a preset premise or assumption, and the inductive query is a query that contains a preset premise or assumption; inputting the non-inductive query and the inductive query into the first large model to obtain a non-inductive response text corresponding to the non-inductive query and an inductive response text corresponding to the inductive query; and determining a hallucination detection result of the initial response text based on the initial response text, the non-inductive response text and the inductive response text. The hallucination detection method based on the inductive query can identify whether the reply is a hallucination reply in real time without test data.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a method, apparatus, device, and chip for hallucination detection of large models. Background Technology

[0002] Large Language Models (LLMs) have achieved remarkable results in Natural Language Processing (NLP) tasks such as intelligent question answering, text generation, and machine translation, thanks to their powerful language understanding and generation capabilities.

[0003] However, LLM often exhibits hallucinations when generating text, which greatly affects the reliability and credibility of LLM-generated content.

[0004] In related technologies, hallucination detection methods typically rely on machine learning with large amounts of training data or manual detection, and the annotation process is time-consuming and labor-intensive. Therefore, a method for hallucination detection using large language models without requiring training data has become a problem that needs to be solved. Summary of the Invention

[0005] This application provides a method, apparatus, device, and chip for hallucination detection of large models, the technical solution of which is as follows: According to one aspect of this application, a method for hallucination detection of large models is provided, the method comprising: The process involves obtaining the input question and the initial response text generated by the first model for the question; generating unguided questions and guided questions based on the question, where unguided questions are those without preset premises or assumptions, and guided questions are those with preset premises or assumptions; inputting the unguided questions and guided questions into the first model to obtain the unguided response text for the unguided questions and the guided response text for the guided questions; and determining the hallucination detection result of the initial response text based on the initial response text, the unguided response text, and the guided response text.

[0006] According to one aspect of this application, a large model illusion detection device is provided, the large model illusion detection device comprising: The receiving module is used to acquire the input question and the initial response text generated by the first model for the question; the control module is used to generate unguided questions and guided questions based on the question, wherein unguided questions are questions that do not contain preset premises or assumptions, and guided questions are questions that contain preset premises or assumptions; the input module is used to input the unguided questions and guided questions into the first model to acquire the unguided response text corresponding to the unguided questions and the guided response text corresponding to the guided questions; the result determination module is used to determine the hallucination detection result of the initial response text based on the initial response text, the unguided response text, and the guided response text.

[0007] According to one aspect of this application, a computer-readable storage medium is provided, wherein at least one program is stored therein, the at least one program being loaded and executed by a processor to implement the illusion detection method for large models described above.

[0008] According to one aspect of this application, a chip is provided that includes programmable logic circuitry and / or program instructions, which, when the chip is run, implement the illusion detection method for large models described above.

[0009] According to one aspect of this application, a computer program product is provided, the computer program product including computer instructions stored in a computer-readable storage medium; a processor reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer program product to implement the illusion detection method for large models described above.

[0010] According to one aspect of this application, a computer program is provided, which is executed by a processor to implement the illusion detection method for large models described in the above aspects.

[0011] The beneficial effects of the technical solution provided in this application include at least the following: The hallucination detection method for large models provided in this application significantly improves the recognition rate and coverage of hallucination content by constructing two methods: one without prompting questions and the other with prompting questions. This avoids the problem of missed detection caused by a single question and ensures that the detection results are comprehensive and reliable.

[0012] Secondly, there is no need to retrain the first major model. Hallucination detection can be completed simply by inputting different types of questions and comparing the responses, achieving low-cost and high-efficiency hallucination detection results. Different tendencies of leading statements can also be adapted to more comprehensive application scenarios. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This application provides a schematic diagram of a device for carrying a large model, as illustrated in an exemplary embodiment. Figure 2 A flowchart of a hallucination detection method for large models provided in an exemplary embodiment of this application is shown; Figure 3 A flowchart illustrating a method for generating non-leading and leading questions based on an exemplary embodiment of this application is shown. Figure 4 A flowchart illustrating a method for obtaining induced responses and non-induced responses provided in an exemplary embodiment of this application is shown. Figure 5 A flowchart of a hallucination detection method for determining the final confidence level provided in an exemplary embodiment of this application is shown; Figure 6 This invention provides an overall flowchart of a hallucination detection method that integrates multiple confidence metrics, according to an exemplary embodiment of this application. Figure 7 A flowchart illustrating a method for obtaining linguistic confidence based on an exemplary embodiment of this application is shown. Figure 8 This invention provides an overall flowchart of a hallucination detection method based on multi-step confidence analysis, according to an exemplary embodiment of this application. Figure 9 A flowchart illustrating a multi-step analysis confidence level acquisition method provided in an exemplary embodiment of this application is shown. Figure 10 This invention provides an overall flowchart of a hallucination detection method based on multi-step confidence analysis, according to an exemplary embodiment of this application. Figure 11 A flowchart illustrating a method for obtaining consistency confidence provided by an exemplary embodiment of this application is shown; Figure 12 A flowchart illustrating the overall process of hallucination detection based on consistency confidence provided in an exemplary embodiment of this application is shown. Figure 13 A flowchart illustrating a method for obtaining the confidence level of a classification model provided in an exemplary embodiment of this application is shown. Figure 14 A schematic diagram of a large-scale hallucination detection device provided in an exemplary embodiment of this application is shown; Figure 15 A schematic block diagram of a processor provided in an exemplary embodiment of this application is shown; Figure 16 A schematic block diagram of a computer device provided in an exemplary embodiment of this application is shown; Figure 17 A schematic block diagram of a computer device provided in an exemplary embodiment of this application is shown; Figure 18 A schematic block diagram of a computer device provided in an exemplary embodiment of this application is shown; Figure 19 A schematic block diagram of a computer device provided in an exemplary embodiment of this application is shown. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0016] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0017] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this disclosure. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0018] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions. For example, the settings and operation information involved in this application were obtained with full authorization.

[0019] It should be noted that the large-scale model for hallucination detection in this application can be deployed on various terminal devices, industry systems and application platforms, without being limited by device type, operating environment and industry field. It can effectively adapt to the usage needs in different scenarios and realize the widespread application of technical and application value.

[0020] For example, such as Figure 1 As shown, the large model can be directly deployed on terminal devices 100, including but not limited to mobile terminals 110, computer web interfaces 120, tablets, smartwatches, smart TVs, and other consumer-grade smart terminals. Model parameters can be dynamically adjusted based on the hardware performance of the terminal devices. In mobile terminals 110, lightweight applications such as intelligent interaction, voice assistants, and content generation can be implemented. In computer web interfaces 120, efficient operations such as complex text processing, data visualization, and intelligent editing can be performed. In terminals such as tablets and smart TVs, personalized needs for multimodal interaction and scene recognition can be adapted, improving the intelligent experience and usage efficiency of the terminal devices. Furthermore, through collaborative deployment between the terminal and the cloud, an organic combination of rapid on-device response and deep cloud-side processing can be achieved, balancing response speed and processing accuracy.

[0021] Furthermore, this large-scale model is adaptable to various deployment modes such as cloud deployment and edge computing deployment. It can be integrated into various applications, system platforms, and third-party services, including but not limited to intelligent customer service systems, content creation platforms, data processing tools, and intelligent security systems. It can meet the daily needs of individual users as well as adapt to the large-scale application needs of enterprises and industries. The model's functional modules and operating parameters can be flexibly adjusted according to the differences in actual application scenarios to achieve efficient application in multiple scenarios and dimensions. This further expands the application boundaries of the large-scale model and has broad industrial application prospects and promotional value.

[0022] It should be understood that although the terms first, second, etc., may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, a first parameter may also be referred to as a second parameter, and similarly, a second parameter may also be referred to as a first parameter. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0023] First, let me introduce the relevant terms used in this application: Hallucination: Hallucination refers to text generated by a model that contains semantic, logical, or factual errors, is inconsistent with real-world knowledge or input context, or generates fictitious content without relevant evidence. For example, in a question-and-answer scenario, the model might incorrectly answer key information such as historical events or historical figures; in text generation tasks, it might fabricate non-existent facts or cite false documents. Hallucinations can be categorized in different ways based on different criteria. According to their relationship to input information, they can be divided into input-related hallucinations and input-irrelevant hallucinations. Input-related hallucinations refer to contradictions or unreasonable extensions between the model-generated content and the input context; input-irrelevant hallucinations refer to content generated by the model that is completely detached from the input and fabricated out of thin air. According to the form of the hallucination content, they can be divided into factual hallucinations and logical hallucinations. Factual hallucinations involve statements that contradict objective facts; logical hallucinations refer to generated content that has flaws in logical reasoning. Fine-tuning large models: Utilizing large-scale pre-trained language models (such as BERT, GPT, etc.), fine-tuning is performed on datasets containing both hallucinatory and non-hallucinatory texts to adapt them to hallucination detection tasks. During fine-tuning, the model learns linguistic features and semantic patterns related to hallucinations, thus gaining the ability to detect hallucinations. For example, using the BERT model, based on pre-training, fine-tuning is performed on a specific hallucination detection dataset to adjust model parameters and improve its performance on the hallucination detection task. The fine-tuned pre-trained language model performs well in hallucination detection accuracy, leveraging the powerful language understanding capabilities of the pre-trained model. However, the fine-tuning process requires significant computational resources and demands high-quality and large-scale datasets. It also faces generalization problems similar to supervised learning, requiring further fine-tuning for new domains or new types of hallucination detection.

[0024] In related technologies, when models generate responses, they may contain content that deviates from factual basis, fabricates false information, or has logical contradictions, and the solutions in these technologies lack the ability to monitor this in real time.

[0025] The common practice in related technologies is to directly feed the generated hallucinatory responses to the front-end interface, where they are then manually or by an external independent system for subsequent verification, review, and judgment. This process not only increases unnecessary system latency but also degrades the overall user experience. Users must wait for the back-end to complete processing before seeing the verification results, resulting in a lengthy interaction chain. Furthermore, front-end review relies on human experience, which is prone to subjective judgment bias and struggles to handle massive, high-concurrency question-and-answer requests, significantly reducing the overall system's response efficiency and operational stability.

[0026] This application proposes a hallucination detection method for large models. It utilizes the model's own generation logic to perform real-time identification and judgment during the question-and-answer process. This reduces response latency, optimizes the user experience, and ensures the system's high efficiency and stability.

[0027] Based on this, this application provides a method, apparatus, device, and chip for hallucination detection of large models.

[0028] It should be noted that the "large model" in this application refers to an artificial intelligence model with a large number of parameters and powerful language understanding and generation capabilities, including but not limited to Large Language Models (LLMs) and multimodal large models. The first and second large models are both specific instances or deployment forms of large models. The first large model serves as the primary model, used to generate initial response text for the input question and respond to both non-leading and leading questions during hallucination detection. The second large model serves as a backup model, used to take over the response generation task and provide alternative response texts when the initial response text generated by the first large model is detected as hallucinating or suspected of being hallucinating. The first and second large models can be models with the same architecture but different parameter scales, or they can be large models with different architectures; this application does not limit this. By setting a primary model and a backup model, the model can be automatically switched when hallucination is detected, improving the robustness and reliability of the system's responses.

[0029] Figure 2 A schematic flowchart of a hallucination detection method for a large model provided in an exemplary embodiment of this application is shown. The method is illustrated by way of example, where it is executed by a large model hallucination detection device. The method includes: Step 210: Obtain the input question and the initial response text generated by the first model for the question.

[0030] The initial response text is the natural language answer generated directly by the first model in response to the input question without any special intervention.

[0031] By obtaining the user's input questions and the initial response text of the first model, the object and standard of hallucination detection are clarified, avoiding detection bias caused by the lack of original data for comparison.

[0032] Step 220: Generate non-leading questions and leading questions based on the question. Non-leading questions are questions that do not contain preset premises or assumptions, while leading questions are questions that contain preset premises or assumptions.

[0033] Unleading questions refer to text that is semantically identical to the original question, neutral in expression, and without any presuppositions or assumptions. In this application, unleading responses are used as a benchmark when input into the first large model to measure the model's output performance under normal and stable conditions. If there is a significant difference between the initial response and the unleading response, it indicates that the model's output is unstable and highly likely to be based on illusion.

[0034] Leading questions refer to adding specific semantic biases or pre-set premises or assumptions to the original question to observe whether the model's output text deviates. These pre-set premises or assumptions include one or more of the following: questioning assumptions (e.g., "Are you sure...?"), reverse perspective assumptions (e.g., "Analyze from the opposite perspective"), conditional assumptions (e.g., "What if...?"), comparative assumptions (e.g., "Compare with other...?"), time / scenario constraint assumptions (e.g., "In the context of..."), and role-playing assumptions (e.g., "From the perspective of..."). In this application, the pre-set premises or assumptions are conditions, viewpoints, scenarios, or logical guidance automatically generated by the first major model when the original question is posed a second time, and are not verified by the model itself. By changing the semantic environment of the question, it detects whether the model's answer will undergo unreasonable variations. The examples listed above are merely illustrative; those skilled in the art can design other equivalent forms of pre-set premises or assumptions without departing from the concept of this application, and this application does not limit such designs.

[0035] Leading statements in leading questions are used to perturb the way questions are asked in order to determine whether the model’s output answer is hallucinatory.

[0036] In some embodiments, the inducing questioning includes one or more of the following: evaluative perturbation that questions accuracy, feedback perturbation that introduces user feedback, conditional perturbation that introduces time or scenario constraints, wording perturbation that changes the form of expression, comparative perturbation that introduces a comparison object, role-playing perturbation that switches perspectives, granular control perturbation that adjusts the level of detail in the answer, interfering perturbation that inserts irrelevant information, domain constraint perturbation that limits the scope of professional knowledge, emotional tendency perturbation that adds emotional color, and deductive perturbation that guides logical reasoning.

[0037] In some embodiments, the presuppositions or assumptions included in the leading questions take one or more of the following forms: Presuppositions that raise questions about accuracy, such as "Is your conclusion reliable? Is there any evidence?"; presuppositions that introduce user feedback, such as "You didn't explain clearly; please provide more specific steps."; presuppositions that introduce time or scenario constraints, such as "Only describe the practice in a campus setting; don't go beyond that scope."; presuppositions that change the wording, such as "Rewrite your answer in a more colloquial and down-to-earth way."; presuppositions that introduce comparative examples, such as "How does this method differ from traditional methods?"; presuppositions that shift perspectives, such as "Reinterpret this concept from a student's point of view."; presuppositions that adjust the granularity of the answer's level of detail, such as "Only state the core points; don't elaborate on details."; and presuppositions that insert no... Distracting presuppositions regarding information, such as "The weather is nice today, what does this knowledge point mean?"; domain-constraining presuppositions limiting the scope of professional knowledge, such as "Explain using only junior high school physics knowledge, don't go beyond the syllabus."; emotionally charged presuppositions, such as "Explain the problems with this plan in a disappointed or complaining tone."; deductive presuppositions guiding logical reasoning, such as "Deduce step by step before giving the final conclusion."; format-conversion presuppositions changing the interaction format, such as "Organize the content you just said into a table before answering."; contextual presuppositions introducing cultural background, such as "Explain this custom using the context of a traditional festival."; and example-guided presuppositions providing reference examples, such as "If someone answers this question incorrectly like '1+1=3,' see if you can copy it." In summary, step 220 addresses the problem of single-detection methods being insufficient to comprehensively detect model illusions by generating two types of questions: unguided questions (without pre-set premises or assumptions) and guided questions (with pre-set premises or assumptions). Furthermore, the stability of the model's responses can be assessed by observing the differences between the various responses obtained from different questioning methods.

[0038] Step 230: Input the unguided question and the guided question into the first large model to obtain the unguided response text for the unguided question and the guided response text for the guided question.

[0039] The unguided response text is the model's response to the unguided question. The unguided response text is combined with the response text of the initial response text to determine their consistency. The consistency result is used to evaluate the stability of the model output.

[0040] In some embodiments, the overlap between the uninduced response and the initial response can be quantified by calculating text similarity. By setting a threshold, if the similarity is below the threshold, it can be preliminarily determined that the output is unstable or that hallucinations exist.

[0041] The induced response is the first model's response to induced questions. The degree of difference between the induced response text and the initial response text can be used to determine whether hallucinations exist.

[0042] In some embodiments, when faced with challenging prompts, the first model will revise its initial response; In some embodiments, when faced with guiding inducements, the first major model will generate incorrect information in accordance with the direction of the inducement.

[0043] By incorporating induced questions and induced responses, and analyzing the variation directions of various induced responses, we can more comprehensively obtain the reliability of the model's output and the accuracy of hallucination detection results.

[0044] In summary, step 230 directly reuses the first main model for reasoning, without introducing other models for hallucination detection, thus reducing the cost and difficulty of hallucination detection. Simultaneously, by acquiring uninduced responses and multiple sets of responses, the differences in output under different questioning scenarios can be used to comprehensively analyze and compare the responses of the first main model to determine whether hallucinations exist.

[0045] Step 240: Determine the hallucination detection result of the initial response text based on the initial response text, the uninduced response text, and the induced response text.

[0046] In some embodiments, if the difference between the initial response text and the uninduced response text is greater than a threshold, it is determined that the output is unstable; if the induced statement successfully induces the model to change the facts or fabricate content, it is determined that a hallucination exists.

[0047] Hallucination test results include: no hallucinations, suspected hallucinations, and hallucinations present.

[0048] In some embodiments, the determination of the hallucination detection result can be used to control subsequent processes.

[0049] For example, if a hallucination is determined to exist, the model can provide a fallback response, introduce the answer from a second major model, or introduce the answer from a knowledge base.

[0050] In summary, by combining the analysis of the initial response text, the uninduced response text, and multiple sets of induced response texts, it is possible to distinguish whether the output of the first major model is stable and whether there is hallucination, thereby improving the accuracy and reliability of the detection results.

[0051] Next, we will introduce the overall process of hallucination detection for large models.

[0052] like Figure 3 As shown, regarding step 220, in some embodiments, generating non-leading questions and leading questions based on the question includes: Step 221: Generate at least one non-leading question and multiple leading questions based on the question, wherein the multiple leading questions each contain different leading statements.

[0053] In some embodiments, the persuasive statement includes a presupposition or assumption.

[0054] In some embodiments, leading statements are used to modify the way questions are asked, causing the first model to generate response text that differs from that without leading questions.

[0055] In some embodiments, leading statements are used to perturb the way a question is asked, including at least one of the following: Evaluative perturbations that challenge accuracy are used to question the veracity of the original response, allowing the model to validate its own output. For example, it might ask, "Is the conclusion you just stated reliable? Is there any evidence to support it?" Introducing feedback perturbations allows the model to supplement or adjust its responses based on user feedback. For example, if you didn't explain clearly earlier, here are the specific steps:

[0056] Conditional perturbations with time or scenario constraints are introduced. By adding time or scenario constraints without changing the content of the problem, these perturbations are used to detect whether the model's output response exceeds the range boundary. For example, only the approach in a campus scenario is described; it should not exceed this range.

[0057] This involves altering the wording and expression, changing only the wording and style. For example, replace the previous answer with a more colloquial and down-to-earth statement.

[0058] By introducing contrastive perturbations to the comparison objects, the responses are enriched through contrast. For example, how does this method differ from traditional methods? The role-playing perturbation involves switching perspectives and adjusting tone and descriptive viewpoints to modify the content of inquiries. For example, reinterpreting the concept from the student's perspective.

[0059] Adjust the granularity of the perturbation to regulate the level of detail in the answer, specifying the required level of detail. For example, only the core points need to be stated, without elaborating on the details.

[0060] Interfering perturbations by inserting irrelevant information into the query to disrupt the model's attention. For example, "The weather is nice today, what does this knowledge point mean?" Domain-specific constraint perturbations, limiting the scope of professional knowledge, are used to constrain the output range of the indication. For example, explanations should be based solely on middle school physics knowledge, avoiding anything beyond the scope of the curriculum.

[0061] By introducing emotional biases and altering the tone and descriptive style, the focus of the test model's response may shift. For example, describe the problems with the solution using a tone of disappointment or complaint.

[0062] Deductive perturbations that guide logical reasoning require responses to include the reasoning process. For example, one might deduce the reasoning step by step before presenting the final conclusion.

[0063] This involves changing the format of the interaction, such as switching from text to a table for the question. For example, you could organize the previous content into a table before answering.

[0064] The text introduces a contextual presupposition perturbation based on cultural background, requiring responses to conform to the cultural context of the input. For example, it uses the context of a traditional festival to explain this custom.

[0065] Provide example-guided perturbations as reference paradigms, requiring the model to mimic these paradigms, which can be incorrect or irrelevant. Test whether the model can identify the errors. For example, consider incorrect answers like "1+1=3," and see if the model will simply copy them.

[0066] like Figure 4 As shown, regarding step 230, in some embodiments, the unleaded question and the led question are input into the first large model to obtain the unleaded response corresponding to the unleaded question and multiple led responses corresponding to the led questions, including: Step 231: Input at least one unguided question and multiple guided questions into the first large model to obtain the unguided response text corresponding to at least one unguided response and the guided response text corresponding to multiple guided responses; wherein, each unguided question corresponds to one unguided response text and each guided question corresponds to one guided response text.

[0067] like Figure 5 As shown, regarding step 240, in some embodiments, the hallucination detection result of the initial response text is determined based on the initial response text, the absence of a induced response, and the induced response, including: Step 241: Based on the initial response text, at least one uninduced response text, and multiple induced response texts, determine the final confidence level used to characterize the reliability of the initial response text.

[0068] In some embodiments, at least two confidence metrics are determined based on the initial response text, the uninduced response text, and multiple induced response texts; the at least two confidence metrics are fused to obtain a confidence level that characterizes the reliability of the initial response text.

[0069] In some embodiments, the confidence metrics include at least two of the following: the linguistic confidence of the first large model for question generation, the multi-step analysis confidence of the first large model for question generation, the consistency confidence between the unguided response text and multiple guided response texts, and the correctness confidence based on the output of the classification model.

[0070] In some embodiments, the process of fusing at least two confidence indices includes one or more of the following: taking the average of the at least two confidence indices, taking the median, taking the average after removing extreme values, and taking the minimum value.

[0071] 1. Fuse at least two confidence levels in the confidence index. The confidence scores from multiple rounds are averaged: the confidence scores from the first round (without inducement) plus all induced rounds are collected, and the average is taken as the final confidence score. For example, the confidence score without inducement is 0.95, and the confidence scores for the three induced rounds are 0.93, 0.88, and 0.90, respectively. The final confidence score is equal to (0.95 + 0.93 + 0.88 + 0.90) / 4, which is equivalent to a final confidence score of 0.915.

[0072] Difference analysis (relatively stable): Calculate the absolute difference between the confidence level without inducement and the confidence levels in each round. First, take the mean of the differences, then subtract the mean difference from the confidence level without inducement (the larger the difference, the less stable the model). For example, with a confidence level of 0.95 without inducement, and confidence levels of 0.93, 0.88, and 0.90 in three rounds with inducement, the mean difference equals (0.02 + 0.07 + 0.05) / 3, which equals 0.047. The final confidence level equals 0.95 minus 0.047, which is 0.903.

[0073] Lowest confidence level (most conservative): The lowest confidence level across all rounds is used as the final confidence level to avoid misjudgments caused by model overconfidence. For example, the uninduced confidence level is 0.95, the induced confidence levels for the three rounds are 0.93, 0.88, and 0.90, and the final confidence level is 0.88.

[0074] Median calculation: Collect the confidence scores from the first round (without inducement) plus all induced rounds. Arrange all confidence scores in ascending order and take the median value as the final confidence score. If the total number of confidence scores is even, take the average of the two median values ​​as the final confidence score. For example, the confidence score without inducement is 0.95, and the confidence scores for the three induced rounds are 0.93, 0.88, and 0.90 respectively. The total number of confidence scores is 0.88, 0.90, 0.93, and 0.95. Since the total number of confidence scores is even, the final confidence score is (0.90 + 0.93) / 2, which is 0.915.

[0075] Remove extreme values ​​(eliminate outlier interference): Collect the confidence scores from the first round (without induction) plus all induction rounds. First, remove the maximum and minimum values ​​from all the values ​​(if multiple identical extreme values ​​exist, remove only one maximum and one minimum value). Then, average the remaining confidence scores to obtain the final confidence score. This effectively eliminates the interference of extreme outliers on the results. For example, the confidence score without induction is 0.95, and the confidence scores for the three rounds of induction are 0.93, 0.88, and 0.90, respectively. The total confidence scores are 0.88, 0.90, 0.93, and 0.95. After removing the maximum value of 0.95 and the minimum value of 0.88, the remaining values ​​are 0.90 and 0.93. The final confidence score is equal to (0.90 + 0.93) / 2, which is 0.915.

[0076] Step 242: Compare the final confidence level with the preset threshold to obtain the comparison result; and obtain the judgment result based on whether there are contradictory responses between the non-leading response text and the leading response text.

[0077] Hallucination test results include at least one of the following: hallucination present, suspected hallucination, and no hallucination.

[0078] In some embodiments, the determination of the hallucination detection result corresponding to the initial response text by comparing the confidence level with a preset threshold and considering whether there are contradictory responses between the non-induced response and the induced response includes at least one of the following methods, wherein the exemplary preset threshold includes a first threshold and / or a second threshold: When the final confidence level is less than the first threshold, the hallucination detection result is determined to be that a hallucination exists; When the final confidence level is greater than or equal to the first threshold and less than the second threshold, the hallucination detection result is determined to be a suspected hallucination. When the final confidence level is greater than or equal to the second threshold and there are no contradictory responses between the no-inducing responses and multiple induced responses, the hallucination detection result is determined to be that there is no hallucination. When there are contradictory responses between no induced response and multiple induced responses, and the final confidence level is less than the second threshold, the hallucination detection result is determined to be hallucination or suspected hallucination. The first threshold is less than the second threshold.

[0079] Step 243: Based on the comparison results and judgment results, determine the hallucination detection result corresponding to the initial response text.

[0080] In some embodiments, if the hallucination detection result indicates the presence of hallucination or suspected hallucination, the input question is input into a second large model to generate the response text of the second large model. The hallucination detection process described in steps 210 to 240 above is then performed on the response text of the second large model. In this case, the process is a secondary hallucination detection.

[0081] For example, if the hallucination detection result indicates that there is no hallucination, the response text of the second largest model is output as the final response result. If the hallucination detection result indicates that there is a hallucination or a suspected hallucination, a preset fallback response or a matching knowledge base answer is output as the final response.

[0082] In some embodiments, when the hallucination detection result indicates the presence or suspected hallucination, a preset fallback response or a matching knowledge base answer is directly provided as the final response, without performing secondary hallucination detection using the second major model.

[0083] Figure 6 An overall flowchart of hallucination detection fused with multiple confidence metrics provided in an exemplary embodiment of this application is shown.

[0084] like Figure 6 As shown, in step 610, input the question. The question is input by the user, and the large model receives the user's input question and provides an initial response.

[0085] Step 620: The first large model generates an initial response. Since the first large model has already generated an initial response, the process enters the hallucination detection phase. In the hallucination detection module, non-leading questions and leading questions relative to the user's input question are generated. Corresponding non-leading questions and leading questions are then generated in the hallucination detection module. Leading questions may include one or more.

[0086] Step 630: Perform confidence estimation. Each response is matched with a confidence level. The confidence levels of non-induced responses and induced responses are summed to obtain a confidence level of induced consistency.

[0087] Step 640: Perform multi-indicator fusion. The confidence indicators include one or more of the following: the linguistic confidence of the first major model in generating the question, the multi-step analysis confidence of the first major model in generating the question, the consistency confidence between no-leading responses and multiple-leading responses, and the correctness confidence based on the output of the classification model.

[0088] The relevant introductions to the above-mentioned "linguistic confidence, multi-step analysis confidence, consistency confidence, and correctness confidence based on classification model output" will be introduced in "2. Linguistic Confidence", "3. Multi-step Analysis Confidence", "4. Consistency Confidence", and "5. Correctness Confidence Based on Classification Model Output".

[0089] Step 650: Two-level threshold determination, comparing the final confidence score obtained by fusing multiple indicators with a preset threshold. The preset threshold includes a first threshold and a second threshold, with the first threshold being less than the second threshold.

[0090] When the final confidence level is less than the first threshold, the hallucination detection result is determined to be hallucination; when the final confidence level is greater than or equal to the first threshold and less than the second threshold, the hallucination detection result is determined to be suspected hallucination; when the final confidence level is greater than or equal to the second threshold and there are no leading responses or contradictory responses among multiple leading response texts, the hallucination detection result is determined to be no hallucination; when there are no leading responses or contradictory responses among multiple leading responses, and the final confidence level is less than the second threshold, the hallucination detection result is determined to be hallucination or suspected hallucination.

[0091] For step 650, the hallucination detection result is obtained. If there is no hallucination, the first major model response is directly output; if there is hallucination or suspected hallucination, the second major model fallback process is executed and a second detection is performed.

[0092] If the hallucination detection result of the second detection is no hallucination, the backup response generated by the second model is output; if the hallucination detection result of the second detection is still suspected hallucination or hallucination exists, the fallback response or the answer matched by the knowledge base is output.

[0093] Next, we will introduce an exemplary embodiment of hallucination detection and judgment using the induced questioning described above: Example 1: Confidence estimation based on induced consistency A user typed: "Does the Earth take 365 days to revolve around the sun?" Different induction methods: The first query (without prompting) was: "Give a confidence level of 0.0 to 1.0 to explain the correctness of 'the Earth revolves around the Sun once in 365 days', and return only the number." The confidence level obtained was 0.95. The second question (questioning the error): "Some argue that the Earth's revolution around the sun is not 365 days. Do you agree that 'the Earth's revolution around the sun is 365 days'? Give a confidence level of 0.0 to 1.0 and return only the number." The answer was 0.94. The third query (from a reverse perspective): "Analyze from the perspective of 'whether the Earth's orbital period is possible but not 365 days', give a confidence level (0.0 to 1.0) for 'the Earth revolves around the sun once in 365 days', and return only the number", resulting in 0.93; The fourth inquiry (accuracy question): "Considering leap years, is the statement 'the Earth revolves around the sun in 365 days' absolutely accurate? Give a confidence level of 0.0 to 1.0 and return only the number," yielding 0.88; The fifth query (comparing with other models): "Other models believe that the Earth revolves around the Sun once in 365.24 days. You provide the confidence level (0.0-1.0) for 'The Earth revolves around the Sun once in 365 days' and return only the number," resulting in 0.89; The sixth question (time-related): "The latest research in this field over the past xxx years indicates that the relevant conclusions may be updated. Does your original answer still hold true? Give a confidence level range of 0.0 to 1.0", and the result is 0.92.

[0094] The final confidence level in this embodiment is calculated by averaging the six values: (0.95+0.94+0.93+0.88+0.89+0.92) / 6 equals 0.918.

[0095] Hallucination determination involves setting several thresholds for three scenarios (clear hallucination, suspected hallucination, and no hallucination). Two thresholds are required: a first threshold and a second threshold, with the first threshold being less than the second threshold.

[0096] Clear hallucinations: those with a final confidence level less than the first threshold and contradictory responses when perturbed; Suspected hallucination: The final confidence level shows contradictory answers between the first and second thresholds and when perturbed; No hallucinations: The final confidence level is greater than the second threshold and there are no contradictory answers when perturbed; In this embodiment, the first threshold is 0.8 and the second threshold is 0.6. The system judges that the final confidence level of 0.918 > 0.8, which is greater than the first threshold, and determines that there is no hallucination (only the details of leap years are not mentioned). The final reply is "The time it takes for the Earth to revolve around the sun is about 365 days (actually 365.24 days, with leap years making up the difference), and this statement is basically accurate (overall confidence level 0.92)".

[0097] 2. Linguistic confidence like Figure 7 As shown, the methods for obtaining linguistic confidence include: Step 710: Input the confidence assessment query into the first model. The confidence assessment query is used to request the first model to self-assess the correctness of the initial response text. Step 720: Obtain the linguistic confidence assessment result output by the first large model. The linguistic confidence assessment result contains a description of the confidence level of the first large model in the initial response text. Step 730: Convert the linguistic confidence assessment results into numerical confidence scores to obtain the linguistic confidence scores.

[0098] This method for calculating confidence is used to avoid the problems of overconfidence or overconservatism in model self-evaluation. It makes targeted corrections to the initially transformed linguistic confidence scores, and the correction strategies include one or more of the following: 1. Correction based on problem type In some embodiments, for common sense and factual objective questions, such as questions related to historical time, scientific principles, and geographical common sense, the model's self-assessment is prone to "overconfidence." If the initial confidence level is ≥0.90, it should be lowered by 0.05-0.10.

[0099] In some embodiments, for subjective analysis and reasoning problems, such as policy interpretation, literary analysis, and logical reasoning-related problems, the model self-assessment is prone to being "overly conservative." If the initial confidence level is ≤0.70, it should be adjusted upward by 0.05-0.10.

[0100] 2. Correction based on the characteristics of the evaluation results In some embodiments, the large model does not specify the evaluation criteria in the linguistic evaluation, but only briefly states the confidence level, with the initial confidence level lowered by 0.05.

[0101] In some embodiments, the model explicitly states the evaluation basis (e.g., "based on XX facts" or "according to XX principle") in the verbal evaluation, and the initial confidence level remains unchanged or is increased by 0.03.

[0102] Figure 8 A flowchart illustrating the overall process of hallucination detection based on multi-step confidence analysis provided in an exemplary embodiment of this application is shown.

[0103] like Figure 8 As shown, there are some differences in the confidence calculation in the hallucination detection module compared to that based on induced questioning. This embodiment only introduces the differences.

[0104] In the hallucination detection module, step 810 is executed to perform a confidence assessment query. The confidence assessment query is then concatenated into the user-input question and input into the large model.

[0105] Execute step 820, and when the first large model generates the initial response, display the confidence level assessment results.

[0106] Step 830: Perform confidence estimation and provide a verbal confidence score based on the confidence assessment results.

[0107] Figure 8 The remaining processes shown refer to " Figure 6 The diagram illustrates the overall flowchart of a hallucination detection system that integrates multiple confidence metrics, provided in an exemplary embodiment of this application. The description in the document is omitted here.

[0108] Next, we will introduce an exemplary embodiment of using the linguistic confidence level described above in hallucination detection: Example 2: Command: Please answer this question and provide your confidence level, ranging from 0% to 100%. The return format is as follows: Answer: XXXX Confidence level: XXXX Set the user's question as an inquiry, concatenate it with the command, and input it into the system.

[0109] For example: User input: What is the capital of China? Input a large model: What is the capital of China? Please answer this question and provide your confidence level, ranging from 0% to 100%. The return format is as follows: Answer: XXXX Confidence level: XXXX Output: Large model display output: Answer: Beijing Confidence level: 95% Example 2-1: Correction based on evaluation result characteristics Original question from the user: What is the capital of China? Initial response from the first major model: Beijing Confidence assessment question: Please assess the correctness of your answer "Beijing" to the question "What is the capital of China?" and express your level of confidence in natural language.

[0110] Model linguistic evaluation results: Very certain to be correct; the content is basic geographical common sense and contains no errors.

[0111] Numerical conversion: This is a very high confidence level, with clear evidence. The core baseline value of 0.95 is taken and no adjustment is needed. The final verbal confidence level is 0.95.

[0112] Example 2-2: Correction based on evaluation result characteristics and problem types Original question from the user: In what year was penicillin discovered? The first major model's initial response: 1929 Confidence assessment question: Please assess the correctness of your answer "1929" to the question "In what year was penicillin discovered?" and express your confidence level in natural language.

[0113] Model linguistic evaluation results: basically confirmed to be correct, most likely 1929, with possible minor time detail deviations.

[0114] Numerical conversion: It belongs to high confidence, mentioning details of the bias, and takes the core baseline value of 0.85; the problem is a factual objective problem, the initial confidence level is <0.90, no correction is needed, and the final verbal confidence level is 0.85.

[0115] Example 2-3: Problem Type-Based Correction User question: How to evaluate a company's market strategy? The initial response from the first major model: The company's market strategy aligns with industry trends, but its channel layout has certain shortcomings.

[0116] Confidence assessment question: Please assess the correctness of your answer to the question "How would you evaluate a company's market strategy?" and express your confidence level in natural language.

[0117] Model language evaluation results: It is uncertain whether it is completely correct; the analysis direction is in line with industry trends; some viewpoints lack specific data support.

[0118] Numerical conversion: It belongs to medium confidence level, mentioning "some content is not supported by data", so we take 0.60; the problem is a subjective analysis type, the initial confidence level is ≤0.70, which is increased by 0.05, and the final verbal confidence level is 0.65.

[0119] In summary, the method for obtaining linguistic confidence is simple to implement, does not require the introduction of external models, directly reuses the self-evaluation capability of the first main model, and has high detection efficiency; it supports flexible query template design and can be adapted to different types of question scenarios.

[0120] 3. Multi-step analysis of confidence levels like Figure 9 As shown, the methods for obtaining confidence levels in multi-step analysis include: Step 910: Input the step decomposition query into the first big model. The step decomposition query is used to request the first big model to decompose the reasoning process of answering the question into multiple steps.

[0121] In some embodiments, when the reasoning process is decomposed into multiple steps, invalid steps that are irrelevant to the question, contain repetitive statements, or have logical breaks need to be eliminated, and only valid steps that directly support the formation of the answer need to be retained. For example, if the user's question is "Why did Zhuge Liang let Cao Cao go at Huarong Road?", if the model steps include "The author of Romance of the Three Kingdoms is Luo Guanzhong", then it is determined to be an invalid step and is eliminated.

[0122] Step 920: Obtain the inference step sequence output by the first large model.

[0123] Step 930: Input the step verification query into the first large model. The step verification query is used to request the first large model to verify the correctness of each step in the inference step sequence.

[0124] Step 940: Obtain the validation results of the first large model for each inference step.

[0125] Step 950: Calculate the confidence level of the multi-step analysis based on the verification results of each inference step in the inference step sequence.

[0126] Figure 10 A flowchart illustrating the overall process of hallucination detection based on multi-step confidence analysis provided in an exemplary embodiment of this application is shown.

[0127] like Figure 10 As shown, there are some differences in the confidence calculation in the hallucination detection module compared to that based on induced questioning. This embodiment only introduces the differences.

[0128] Step 1010 requires the model to break down the reasoning process of answering user-input questions into multiple steps.

[0129] In the hallucination detection module, step 1020 is executed, and step verification queries are performed. The step verification queries are concatenated into each step and input into the large model.

[0130] Execute step 1030 to provide the verification results for each inference step.

[0131] Step 1040: Perform confidence estimation and provide a multi-step confidence analysis based on the confidence assessment results.

[0132] Figure 10 The remaining processes shown refer to " Figure 6 The diagram illustrates the overall flowchart of a hallucination detection system that integrates multiple confidence metrics, provided in an exemplary embodiment of this application. The description in the document is omitted here.

[0133] Example 3: Confidence level of multi-step analysis Design your own instructions, including: Please answer this question and provide the steps of the analysis process, along with a confidence level for each step, ranging from 0% to 100%.

[0134] The return format is as follows: Reply to the answer Analysis process: Step 1: XXXX Confidence level: XXX Step 2: XXXX Confidence level: XXX ... Similarly, the user's questions and instructions are concatenated and input together into the first main model.

[0135] Example 3-1: User input: In the Romance of the Three Kingdoms, why did Zhuge Liang deliberately let Cao Cao escape at Huarong Road during the Battle of Red Cliffs? →Instructions recognized by the large model: Input: In the Romance of the Three Kingdoms, during the Battle of Red Cliffs, why did Zhuge Liang deliberately let Cao Cao escape at Huarong Road? Please answer this question, provide the steps of your analysis, and assign a confidence level to each step, ranging from 0% to 100%.

[0136] The return format is as follows: Reply to the answer Analysis process: Step 1: XXXX Confidence level: XXX Step 2: XXXX Confidence level: XXX ... Output: Zhuge Liang's decision to let Cao Cao go at Huarong Road was the result of multiple factors, including strategic considerations (to contain Eastern Wu), managing personal relationships (to repay Guan Yu's kindness), and establishing his authority internally, and was packaged as "divine will."

[0137] The historical record does not show that Cao Cao was released at Huarong Road, but this plot device in the novel reflects Zhuge Liang's far-sighted political wisdom.

[0138] Analysis process: Step 1: The strategic necessity of maintaining the tripartite balance of power. If Cao Cao had been killed after the Battle of Red Cliffs, the north would have been in chaos, but Sun Quan of Eastern Wu would have taken the opportunity to move north and occupy the Central Plains. Liu Bei's strength was weak at the time, and he would have been unable to resist Sun Quan, and might have been annexed by Sun Quan.

[0139] Confidence level: 95% Step 2: Let Guan Yu repay Cao Cao's favor, avoiding the need for future reciprocation on the battlefield. Guan Yu had been treated generously by Cao Cao (gifted the Red Hare horse, granted a marquisate, and allowed free passage through five passes), and had always wanted to repay this favor. By having him release the non-threatening remnants of the defeated army under control, the favor was repaid, and Guan Yu could serve Liu Bei without any future entanglements.

[0140] Confidence level: 90% Step 3: Consolidate Zhuge Liang's prestige within the army. When Zhuge Liang first entered public service, Guan Yu and Zhang Fei were initially unconvinced. Zhuge Liang, through celestial observation, predicted Cao Cao's survival and deliberately instructed Guan Yu to release someone. Later, Guan Yu disobeyed the order and was sentenced to death. Zhuge Liang feigned to enforce military law, but Liu Bei pleaded for his life. This not only made Guan Yu grateful for the mercy shown to him but also demonstrated Zhuge Liang's brilliant strategies and unwavering enforcement of the law, thus establishing his authority.

[0141] Confidence level: 85% Step 4: Literary fatalism that aligns with "divine will". The Romance of the Three Kingdoms is deeply imbued with the idea of ​​"destiny". Zhuge Liang uses "observing celestial phenomena" to explain certain strategic decisions, giving the characters' actions a fatalistic quality and increasing the novel's dramatic effect and mythologizing the strategist's image.

[0142] Confidence level: 80% Step 5: Avoid the moral and political pressure of Liu Bei's group directly killing Cao Cao. If Liu Bei killed Cao Cao, he might be seen as "harming a virtuous man" and a power struggle among loyal ministers of the Han Dynasty (Cao Cao was still the Chancellor of Han at the time). However, by letting the defeated Cao Cao go, Liu Bei could maintain his reputation for "benevolence and righteousness".

[0143] Confidence level: 75% The confidence score is obtained by averaging the confidence scores of each step.

[0144] In some embodiments, if the confidence value in the analysis step is less than a first threshold, it is determined to be a hallucination, and a fallback response, no response, or the generation result of the second largest model is given.

[0145] In some embodiments, if the confidence level in an analysis step is detected to be less than a first threshold, the response is not terminated directly. Instead, the entire analysis process and the confidence level calculation for each step are completed before subsequent processing is performed.

[0146] In summary, breaking down the confidence level of the overall answer into step-level confidence levels allows for precise identification of the specific reasoning steps that lead to the illusion. Visualizing these reasoning steps also facilitates subsequent manual review and model optimization.

[0147] 4. Consistency confidence like Figure 11 As shown, the methods for obtaining consistency confidence include: Step 1110: Calculate the semantic similarity between the text without prompting a response and each prompting response text to obtain multiple similarity values.

[0148] Step 1120: Calculate the consistency confidence score based on multiple similarity values.

[0149] Among them, consistency confidence is positively correlated with similarity value.

[0150] Next, we will introduce an exemplary embodiment of using the consistency confidence estimation described above for hallucination detection and judgment: For the same question, multiple queries are made, including both non-leading and / or leading questions, and the queries are similar to each other. The confidence scores are then averaged to obtain a confidence score.

[0151] Figure 12 A flowchart illustrating the overall process of hallucination detection based on consistency confidence provided in an exemplary embodiment of this application is shown.

[0152] like Figure 12 As shown, there are some differences in the confidence calculation in the hallucination detection module compared to that based on induced questioning. This embodiment only introduces the differences.

[0153] In the hallucination detection module, step 1210 is executed to calculate semantic similarity for multiple induced responses and non-induced responses. Consistency similarity is obtained based on semantic similarity.

[0154] Figure 12 The remaining processes shown refer to " Figure 6 The diagram illustrates the overall flowchart of a hallucination detection system that integrates multiple confidence metrics, provided in an exemplary embodiment of this application. The description in the document is omitted here.

[0155] Example 4: Multiple questions were asked without leading questions. A user typed: "Was Cao Xueqin the author of 'Dream of the Red Chamber'?" System operation: Ask the same question 5 times in a row, each time prompting "Give a probability of 0.0 to 1.0 to explain the correctness of 'The author of Dream of the Red Chamber is Cao Xueqin', and only return a number", and get 5 confidence levels: 0.98, 0.99, 0.97, 0.96, 0.99.

[0156] Confidence level processing: Average value method: (0.98+0.99+0.97+0.96+0.99) / 5=0.978; Median method: After sorting, take the median value, 0.98; Extreme value removal method: Remove the highest value of 0.99 and the lowest value of 0.96, and the average value is (0.98+0.99+0.97) / 3=0.98.

[0157] The system determined that all three processing results were greater than 0.8, indicating no hallucinations, and finally replied, "The main author of 'Dream of the Red Chamber' is Cao Xueqin, and this statement is accurate (overall confidence level 0.98)."

[0158] In summary, this reflects the stability of the model's response to the problem.

[0159] 5. Confidence level of correctness based on the classification model output like Figure 13 As shown, the methods for obtaining the confidence score of a classification model include: Step 1310: Input the initial response text and the question into a pre-trained classification model, which is used to identify hallucinatory content in the output of the large model.

[0160] Step 1320: Obtain the classification result output by the classification model. The classification result includes the probability or confidence score that the initial response text belongs to hallucination.

[0161] Step 1330: Use the probability or confidence score in the classification results as the confidence score of the classification model.

[0162] The classification model serves as an auxiliary detection tool, used for objective hallucination identification of the initial responses from the primary model. A small model suitable for text classification is chosen, eliminating the need for training a complex, large model and reducing deployment and usage costs. The classification model can be a small model using machine learning or deep learning, such as: BERT (Deep Learning - Transformer class) is the mainstream deep learning model for text classification. It is designed specifically for natural language processing, and its core advantages in hallucination detection classification perfectly match the task requirements.

[0163] GBDT (Traditional Machine Learning - Ensemble Tree) is a gradient boosting decision tree, belonging to the ensemble model of traditional machine learning. Its core advantages in classification are flexible fusion of multiple types of features, easy interpretation, and fast training.

[0164] We train a binary classification model to determine whether a question and answer match using open-source question-and-answer datasets. The text vectors for both the question and answer are obtained from the output of the larger model.

[0165] Next, we will introduce an exemplary embodiment of hallucination detection and judgment using the classification model estimation described above: Example 5: Estimation based on classification model Enter: "Did Qin Shi Huang unify the six kingdoms in 221 BC?" Model: Extract the judgment item: "Qin Shi Huang unified the six kingdoms in 221 BC"; Input the judgment terms into the trained binary classification model (the training data consists of historical fact questions and whether the labels are correct). The classification model outputs: "Correct (confidence level 0.96)".

[0166] The system determined that the model was correct and there was no illusion. The final reply was: "The time when Qin Shi Huang unified the six kingdoms was indeed 221 BC."

[0167] The large model is used to perform binary classification on the user's questions and the large model's responses to determine whether it is a hallucination.

[0168] In summary, the introduction of an external objective classification model avoids the bias of the first model's self-evaluation, achieving "third-party verification" and improving the objectivity of confidence. The lightweight classification model has a fast inference speed and can run in parallel with the first model's detection process without affecting the overall detection efficiency. The model can be reused after training, making it suitable for large-scale, high-concurrency hallucination detection scenarios.

[0169] The five confidence calculation methods for large-scale hallucination detection described above can be implemented individually or in combination, and this application does not limit the implementation of these methods.

[0170] The following describes an example of a combined implementation.

[0171] Example 6: Induced Consistency + Classification Model + Step-by-Step Analysis Scenario: A user asks, "Was Fleming the discoverer of penicillin?" (Accuracy is required; this is suitable for inducing consistency and assisting classification models in making judgments.) Implementation steps: Step 1: Preliminary judgment by the classification model System operation: Input "Penicillin discoverer = Fleming" into the classification model, the model outputs "Correct (confidence level of 0.97)", and it is preliminarily determined that there is no hallucination.

[0172] Step 2: Inducing Consistency Verification Three consecutive questions + leading statements: No inducement: confidence level is 0.98; The leading question, "Some data shows that penicillin was discovered by multiple people. Do you still insist that 'Fleming discovered penicillin'?", has a confidence level of 0.96. The leading statement “Other models believe that Fleming and Florey were the discoverers of penicillin, giving a confidence level of 'Fleming was the discoverer of penicillin'” has a confidence level of 0.95. Mean calculation: (0.98+0.96+0.95) / 3 equals 0.963, further confirming no hallucinations.

[0173] Step 3: Step-by-step analysis and supplementary evidence The model is analyzed step by step, and the confidence level for each step is given: Fleming discovered the antibacterial activity of penicillin mold in 1928 with a confidence level of 0.99; Florey et al. subsequently pushed forward the mass production of penicillin, but the core discoverer was Fleming, with a confidence level of 0.97; The final confidence level is 0.98.

[0174] Output: All three methods yielded results >0.95, indicating no hallucinations. The reply was: "The core discoverer of penicillin was Alexander." Fleming (who discovered its antibacterial properties in 1928) and subsequent researchers advanced its mass production, a statement that is accurate (overall confidence level 0.97).

[0175] The question-and-answer system proposed in this application returns the answer to the user when no hallucination is detected.

[0176] In some embodiments, if a hallucination is detected, a second large model is used to generate an answer to see if there is a hallucination. If there is no hallucination, the answer is returned to the user. If there is a hallucination, a fallback answer can be returned, or the answer can be matched with the knowledge base.

[0177] For example, a fallback response might include: "Sorry, the answer generated by the current model may contain factual errors, and we cannot provide you with an accurate result at this time. You can try rephrasing your question, or wait for verification before replying."

[0178] In some embodiments, if a hallucination is detected, a fallback response is returned directly without going through a second hallucination detection in the second major model.

[0179] Based on the method for detecting large-scale hallucinations described in this application, this application tested it on 1000 pieces of common sense data: The data sample is as follows: 1. What is the highest mountain in the world? Mount Everest 2. What is the longest river in the world? The Nile River. 3. In the *Records of the Three Kingdoms*, who killed Hua Xiong? Sun Jian. 4. In *Romance of the Three Kingdoms*, who killed Hua Xiong? Guan Yu. ... Using these 1000 data points for testing, the accuracy improved from 82% to 90%; compared to the fine-tuned large model in related technologies, the accuracy improved from 82% to 88%. In summary, the method proposed in this application can further reduce the output of hallucination responses from large models.

[0180] Next, we will introduce the device side: Figure 14 A schematic diagram of a large model illusion detection device 1400 provided in an exemplary embodiment of this application is shown. The large model illusion detection device 1400 includes: a receiving module 1410, a control module 1420, an input module 1430, and a result determination module 1440.

[0181] Optionally, the receiving module 1410 can be understood as a data acquisition module, the control module 1420 can be understood as both an inquiry generation module and a confidence calculation module, the input module 1430 can be understood as a model interaction module, and the result determination module 1440 can be understood as an illusion detection decision module, or the confidence fusion module can be understood as a part of the result determination module 1440. This application does not limit this.

[0182] The receiving module 1410 is used to acquire the input question and the initial response text generated by the first model for the question.

[0183] In some embodiments, the receiving module 1410 is further configured to perform text preprocessing on the acquired question and initial response text, including text denoising, format standardization, and key information extraction, to form standardized detection base data.

[0184] In some embodiments, the receiving module 1410 is further configured to store the standardized question and initial response text, establish a detection data archive, and provide data support for subsequent multi-round inquiries and confidence calculations.

[0185] The control module 1420 is used to generate non-leading questions and leading questions based on the question. Non-leading questions are questions that do not contain preset premises or assumptions, while leading questions are questions that contain preset premises or assumptions.

[0186] In some embodiments, the control module 1420 is further configured to match a suitable inducement form from a preset preset premise or hypothesis library according to the type of question, and generate an inducement question of the corresponding type.

[0187] In some embodiments, the control module 1420 is further configured to generate at least one non-leading question and multiple leading questions based on the question, wherein the multiple leading questions each have different leading expressions, and the leading expressions are used to perturb the way the question is asked.

[0188] In some embodiments, the control module 1420 is further configured to perform validity checks on the generated non-leading questions and leading questions, and to remove question content that deviates from the core semantics of the original question or whose leading expressions are invalid, so as to ensure the validity of the questions.

[0189] In some embodiments, the control module 1420 is further configured to determine a final confidence level for characterizing the reliability of the initial response text based on the initial response text, the uninduced response, and multiple induced responses.

[0190] In some embodiments, the control module 1420 is further configured to determine at least two confidence metrics based on the initial response text, the uninduced response text, and multiple induced response texts. The confidence metrics include at least two of the following: linguistic confidence, multi-step analysis confidence, consistency confidence, and correctness confidence based on the output of the classification model.

[0191] In some embodiments, the control module 1420 is further configured to perform fusion processing on at least two confidence indicators, including at least one of the following methods: taking the average, taking the median, taking the average after removing extreme values, and taking the minimum value, to obtain the final confidence level.

[0192] Input module 1430 is used to input non-leading questions and leading questions into the first large model to obtain non-leading responses to non-leading questions and leading responses to leading questions.

[0193] In some embodiments, the input module 1430 is further configured to input non-leading queries and multiple leading queries into the first large model in a preset order, ensuring that each input is an independent request and avoiding the model being affected by the previous round of output.

[0194] In some embodiments, the input module 1430 is also used to transmit the text of no inducement response and the text of inducement response to the control module 1420 to provide data support for the calculation of confidence index and the fusion of final confidence.

[0195] The result determination module 1440 is used to compare the final confidence level with a preset threshold, and determine the hallucination detection result corresponding to the initial response text by combining whether there are contradictory responses between the text without induced response and multiple induced response texts.

[0196] In some embodiments, the result determination module 1440 is further configured to preset a first threshold and a second threshold, wherein the first threshold is less than the second threshold, and to divide the hallucination detection result into intervals based on the comparison result between the final confidence level and the threshold.

[0197] In some embodiments, the result determination module 1440 is further configured to perform semantic similarity calculation on the text without induced response and multiple induced response texts, determine whether there is a core information contradiction or a detail contradiction between the response texts, and form a contradiction determination result.

[0198] In some embodiments, the result determination module 1440 is further configured to combine the threshold comparison result of the final confidence level with the response inconsistency determination result to determine the final hallucination detection result, wherein the hallucination detection result includes at least one of the following: hallucination present, suspected hallucination, and no hallucination.

[0199] In some embodiments, the result determination module 1440 is further configured to trigger a fallback processing procedure when the hallucination detection result indicates the presence or suspected hallucination, to transmit the input question to the second major model, or to call the preset knowledge base answer and fallback response.

[0200] In some embodiments, the result determination module 1440 is further configured to output the initial response text of the first large model as the final response result when the hallucination detection result is that there is no hallucination; output the response text of the second large model as the final response result when the hallucination detection result is that there is no hallucination; and output a preset fallback response or a matching knowledge base answer as the final response result when the hallucination detection result is that there is hallucination or suspected hallucination.

[0201] For specific details regarding large-scale hallucination detection, please refer to the relevant descriptions in the above method embodiments, and / or Figures 2 to 13 The relevant descriptions shown will not be repeated here.

[0202] This application provides a processor. Figure 15A schematic block diagram of a processor provided in an exemplary embodiment of this application is shown. The processor 1500 includes the large model illusion detection device provided in the above embodiments, or, embodiments of this application provide a processor including programmable logic circuitry and / or program instructions, which, when running on a computer device, are used to implement the large model illusion detection method provided in the above method embodiments.

[0203] This application provides a computer device. Figure 16 A schematic block diagram of a computer device provided in an exemplary embodiment of this application is shown. The computer device 1600 includes the large model illusion detection apparatus provided in the above embodiments. Alternatively, embodiments of this application provide a computer device for implementing the large model illusion detection method provided in the above method embodiments.

[0204] Figure 17 A schematic block diagram of a computer device provided in an exemplary embodiment of this application is shown. The computer device 1700 includes the processor provided in the above embodiments, and the computer device can be implemented as a terminal. Alternatively, an embodiment of this application provides a computer device for implementing the hallucination detection method for large models provided in the above method embodiments.

[0205] Optionally, embodiments of this application also provide a computer device, the computer device including: a processor and a memory, the memory storing a computer program; the processor is used to execute the computer program in the memory to implement the illusion detection method for large models provided in the above method embodiments.

[0206] Figure 18 A schematic block diagram of a computer device provided in an exemplary embodiment of this application is shown. The computer device is a server 1800. Typically, the server 1800 includes a processor 1801 and a memory 1802.

[0207] Processor 1801 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1801 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). Processor 1801 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1801 may be a Graphics Processing Unit (GPU) with integrated shaders, which is responsible for rendering and drawing the content required to be displayed on the screen. In some embodiments, processor 1801 may also include an Artificial Intelligence (AI) processor, which is used to handle computational operations related to machine learning.

[0208] Memory 1802 may include one or more computer-readable storage media, which may be non-transitory. Memory 1802 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in memory 1802 is used to store at least one instruction, which is executed by processor 1801 to implement the hallucination detection method for large models provided in the method embodiments of this application.

[0209] In some embodiments, server 1800 may optionally include an input interface 1803 and an output interface 1804. Processor 1801, memory 1802, and input interfaces 1803 and 1804 can be connected via a bus or signal lines. Various peripheral devices can be connected to input interfaces 1803 and 1804 via a bus, signal lines, or a circuit board. Input interfaces 1803 and 1804 can be used to connect at least one input / output (I / O) related peripheral device to processor 1801 and memory 1802. In some embodiments, processor 1801, memory 1802, and input interfaces 1803 and 1804 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 1801, memory 1802, and input interfaces 1803 and 1804 can be implemented on separate chips or circuit boards, and this application embodiment does not limit this.

[0210] Figure 19 A schematic block diagram of a computer device provided in an exemplary embodiment of this application is shown. Optionally, the computer device is a terminal 1900.

[0211] The computer device 1900 may be a portable mobile terminal, also referred to as a mobile terminal in this embodiment. Examples include smartphones, tablets, MP3 players (Moving Picture Experts Group Audio Layer III), and MP4 players (Moving Picture Experts Group Audio Layer IV). The computer device 1900 may also be referred to as user equipment, portable terminal, or other names.

[0212] Typically, computer device 1900 includes a processor 1901 and a memory 1902.

[0213] Processor 1901 may include one or more processing cores, such as a 4-core processor or an 8-core processor. Processor 1901 may be implemented using at least one hardware form of DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), or PLA (Programmable Logic Array). Processor 1901 may also include a main processor and a coprocessor. The main processor, also known as the CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state.

[0214] In some embodiments, processor 1901 may be a processor that integrates a GPU (Graphics Processing Unit, shader), which is responsible for rendering and drawing the content that the display screen needs to show. In some embodiments, processor 1901 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0215] The memory 1902 may include one or more computer-readable storage media, which may be tangible and non-transitory. The memory 1902 may also include high-speed random access memory devices and non-volatile storage devices, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1902 are used to store at least one instruction, which is executed by the processor 1901 to implement the hallucination detection method for large models provided in the various method embodiments of this application.

[0216] This application provides a computer-readable storage medium storing a computer program, which is loaded and executed by a processor to implement the illusion detection method for large models provided in the above-described method embodiments.

[0217] This application provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the processor of the computer device to load and execute the illusion detection method for large models provided in the above-described method embodiments.

[0218] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0219] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0220] Those skilled in the art will recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0221] The above are merely optional embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0222] It should be understood that "multiple" as used herein refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, the step numbers described herein are merely illustrative of one possible execution order. In some other embodiments, the steps may not be executed in numerical order, such as two steps with different numbers being executed simultaneously, or two steps with different numbers being executed in the reverse order of the illustration. This application does not limit this.

[0223] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for hallucination detection in large models, characterized in that, The method includes: Obtain the input question and the initial response text generated by the first major model for the question; Based on the question, unleading questions and leading questions are generated, wherein the unleading questions are questions that do not contain any preset premises or assumptions, and the leading questions are questions that contain the preset premises or assumptions; Input the unguided question and the guided question into the first large model to obtain the unguided response text corresponding to the unguided question and the guided response text corresponding to the guided question; Based on the initial response text, the uninduced response text, and the induced response text, the hallucination detection result of the initial response text is determined.

2. The method according to claim 1, characterized in that, The generation of non-leading questions and leading questions based on the question includes: Based on the question, at least one unleading question and multiple leading questions are generated, wherein the multiple leading questions each contain different leading statements; The step of inputting the unleaded question and the led question into the first large model to obtain the unleaded response text corresponding to the unleaded question and the led response text corresponding to the led question includes: Input at least one of the unleading questions and multiple leading questions into the first large model to obtain the unleading response text corresponding to at least one of the unleading questions and the leading response text corresponding to multiple leading questions; Each of the unleaded questions corresponds to an unleaded response text, and each of the led questions corresponds to a led response text.

3. The method according to claim 2, characterized in that, The leading statement is used to change the way the question is asked, so that the first large model generates a response text that is different from the one without leading questions.

4. The method according to any one of claims 1 to 3, characterized in that, The process of determining the hallucination detection result of the initial response text based on the initial response text, the uninduced response text, and the induced response text includes: Based on the initial response text, at least one of the uninduced response texts, and multiple induced response texts, a final confidence level is determined to characterize the reliability of the initial response text; The final confidence level is compared with a preset threshold to obtain a comparison result; and a judgment result is obtained based on whether there are contradictory responses between the non-inducing response text and the induced response text. Based on the comparison results and the judgment results, the hallucination detection result corresponding to the initial response text is determined.

5. The method according to claim 4, characterized in that, The determination of the final confidence level, used to characterize the reliability of the initial response text, based on the initial response text, the uninduced response text, and the induced response text, includes: At least two confidence metrics are determined based on the initial response text, at least one of the uninduced response texts, and multiple induced response texts. The at least two confidence metrics are fused to obtain the final confidence score used to characterize the reliability of the initial response text.

6. The method according to claim 5, characterized in that, The confidence metrics include at least two of the following: The confidence scores are: the linguistic confidence score generated by the first large model for the question, the multi-step analysis confidence score generated by the first large model for the question, the consistency confidence score between the unguided response text and the guided response text, and the correctness confidence score based on the output of the classification model.

7. The method according to claim 5, characterized in that, The fusion processing of the at least two confidence indices includes at least one of the following: The average, median, average after removing extreme values, and minimum value are taken for the at least two confidence indices.

8. The method according to claim 4, characterized in that, The hallucination detection results include at least one of the following: hallucination present, suspected hallucination, and no hallucination. The step of determining the hallucination detection result corresponding to the initial response text based on the comparison result and the judgment result includes at least one of the following: When the final confidence level is less than the first threshold, the hallucination detection result is determined to be that a hallucination exists; When the final confidence level is greater than or equal to the first threshold and less than the second threshold, the hallucination detection result is determined to be a suspected hallucination; When the final confidence level is greater than or equal to the second threshold and there are no contradictory responses between the text without induced response and the text with induced response, the hallucination detection result is determined to be that there is no hallucination. When there are contradictory responses between the unguided response text and the guided response text, and the final confidence level is less than the second threshold, the hallucination detection result is determined to be hallucination or suspected hallucination. Wherein, the first threshold is less than the second threshold.

9. The method according to claim 8, characterized in that, The method further includes: In response to the hallucination detection result indicating the presence or suspected hallucination, the input question is fed into the second model to generate the response text for the second model. Perform secondary hallucination detection on the response text of the second large model; When the secondary hallucination detection result indicates that there is no hallucination, the second large model response text is output as the final response result.

10. The method according to claim 9, characterized in that, The method further includes: When the secondary hallucination detection result indicates the presence or suspected hallucination, a preset response or a matching knowledge base answer is output as the final response result.

11. A large-scale hallucination detection device, characterized in that, The large-scale hallucination detection device includes: The receiving module is used to acquire the input question and the initial response text generated by the first model for the question; A control module is used to generate unleading questions and leading questions based on the question, wherein the unleading questions are questions that do not contain preset premises or assumptions, and the leading questions are questions that contain the preset premises or assumptions; The input module is used to input the unguided question and the guided question into the first large model to obtain the unguided response text corresponding to the unguided question and the guided response text corresponding to the guided question; The result determination module is used to determine the hallucination detection result of the initial response text based on the initial response text, the non-induced response text, and the induced response text.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one program, which is loaded and executed by a processor to implement the hallucination detection method for large models as described in any one of claims 1 to 10.

13. A chip, characterized in that, The chip includes programmable logic circuitry and / or program instructions, which, when the chip is running, implement the illusion detection method for large models as described in any one of claims 1 to 10.

14. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium; the processor of the computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to implement the hallucination detection method for large models as described in any one of claims 1 to 10.

15. A computer program, characterized in that, The computer program is executed by a processor to implement the hallucination detection method for large models as described in any one of claims 1 to 10.

Images