Method and device for correcting hallucination evaluation data, electronic equipment and storage medium

Abstract

This application provides a method, apparatus, electronic device, and storage medium for correcting hallucination assessment data, relating to the field of computer technology, and is used to improve the reliability of hallucination assessment data. The method includes: generating hallucination assessment data for a target domain based on knowledge documents of that target domain; validating the hallucination assessment data based on a verification model set; and correcting invalid items in the hallucination assessment data if invalid items exist.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to methods, apparatus, electronic devices and storage media for correcting hallucination assessment data. Background Technology

[0002] The illusion phenomenon in large language models refers to the generation of content that is inconsistent with reality, logically contradictory, or deviates from the context. In the field of large language model illusion research, illusion evaluation datasets play a crucial role, and their quality directly affects the accuracy and reliability of the evaluation of model illusion phenomena.

[0003] Current methods for generating hallucination assessment data primarily rely on builders with domain-specific knowledge designing questions based on established knowledge sources and writing corresponding responses that contain factual errors.

[0004] However, the current approach may render the generated hallucination assessment data unreliable, affecting the reliability of the hallucination assessment data. Summary of the Invention

[0005] This application provides a method, apparatus, electronic device, and storage medium for correcting hallucination assessment data, which improves the reliability of hallucination assessment data.

[0006] In a first aspect, this application provides a method for correcting hallucination assessment data, comprising: generating hallucination assessment data for the target domain based on knowledge documents of the target domain; the hallucination assessment data including hallucinogenic questions, correct responses, hallucination responses, and hallucination type labels corresponding to the hallucination responses; validating the hallucination assessment data based on a set of validation models; the set of validation models including multiple validation models with different structures; and correcting invalid items when invalid items exist in the hallucination assessment data.

[0007] The technical solution provided in this application offers at least the following benefits: By generating hallucination assessment data from knowledge documents in the target domain, it ensures a high degree of relevance between the data and the specific professional field, as well as the accuracy of its factual basis, providing high-quality domain-adapted samples. The validity of the hallucination assessment data is verified using a set of verification models containing multiple verification models with different structures, reducing the risk of missed or incorrect judgments due to the limitations of a single model. Furthermore, targeted corrections are made to identified invalid items, enabling iterative optimization of the hallucination assessment data quality. This allows for targeted calibration and reconstruction of hallucination assessment data that does not conform to preset rules and hallucination types, thereby improving the quality and reliability of the hallucination assessment data.

[0008] One possible implementation is that the verification model set includes at least one of the following: a hallucination question verification model set, a correct response verification model set, a hallucination response verification model set, and a hallucination label verification model set; the hallucination question verification model is used to verify whether the hallucination question meets the preset hallucination question generation rules; the correct response verification model is used to verify whether the correct response meets the preset correct response generation rules; the hallucination response verification model is used to verify whether the hallucination response meets the preset hallucination response generation rules; and the hallucination label verification model is used to verify whether the hallucination type label and the hallucination response have consistent characteristics.

[0009] Another possible implementation is to correct invalid items based on a correction model; wherein the correction model includes at least one of the following: hallucination problem correction model, correct response correction model, hallucination response correction model, and hallucination label correction model.

[0010] Another possible implementation involves extracting at least one knowledge triple from a knowledge document in the target domain. The knowledge triple includes a type of domain knowledge from the knowledge document, an entity from the domain knowledge, and an identifier from the target domain. The domain knowledge is factual information from the knowledge document. Based on the knowledge triple, illusion assessment data is generated.

[0011] Another possible implementation is to correct invalid items based on knowledge triples when the invalid item is a hallucinogenic question; to correct invalid items based on knowledge triples and the hallucinogenic question when the invalid item is a correct response; to correct invalid items based on knowledge triples, the hallucinogenic question, and the correct response when the invalid item is a hallucination response; and to correct invalid items based on knowledge triples, the hallucinogenic question, the correct response, and the hallucination response when the invalid item is a hallucination type label.

[0012] Another possible implementation is to verify the validity of the correction result; if the correction result is valid, the correction ends; if the correction result is invalid, the correction is iterated; wherein, the iterative correction includes iteratively correcting the previous correction result.

[0013] Secondly, this application provides a device for correcting hallucination assessment data, comprising: a generation module and a processing module; the generation module is used to generate hallucination assessment data for a target domain based on knowledge documents of the target domain; the processing module is used to verify the validity of the hallucination assessment data based on a set of verification models; the set of verification models includes multiple verification models with different structures; and the invalid items are corrected when there are invalid items in the hallucination assessment data.

[0014] One possible implementation is that the verification model set includes at least one of the following: a hallucination question verification model set, a correct response verification model set, a hallucination response verification model set, and a hallucination label verification model set; the hallucination question verification model is used to verify whether the hallucination question meets the preset hallucination question generation rules; the correct response verification model is used to verify whether the correct response meets the preset correct response generation rules; the hallucination response verification model is used to verify whether the hallucination response meets the preset hallucination response generation rules; and the hallucination label verification model is used to verify whether the hallucination type label and the hallucination response have consistent characteristics.

[0015] Another possible implementation is that the processing module is also used to correct invalid items based on a correction model; wherein the correction model includes at least one of the following: a hallucination problem correction model, a correct response correction model, a hallucination response correction model, and a hallucination label correction model.

[0016] Another possible implementation involves a generation module that extracts at least one knowledge triple based on a knowledge document of the target domain. The knowledge triple includes a type of domain knowledge in the knowledge document, an entity in the domain knowledge, and an identifier of the target domain. The domain knowledge is factual information in the knowledge document. Based on the knowledge triple, illusion assessment data is generated.

[0017] Another possible implementation, the processing module, is also used to correct invalid items based on knowledge triples when the invalid item is a hallucinogenic question; to correct invalid items based on knowledge triples and the hallucinogenic question when the invalid item is a correct response; to correct invalid items based on knowledge triples, the hallucinogenic question, and the correct response when the invalid item is a hallucination response; and to correct invalid items based on knowledge triples, the hallucinogenic question, the correct response, and the hallucination response when the invalid item is a hallucination type label.

[0018] Another possible implementation is that the processing module is also used to verify the validity of the correction result; if the correction result is verified to be valid, the correction ends; if the correction result is verified to be invalid, the correction is iterated; wherein, the iterative correction includes iterative correction of the previous correction result.

[0019] Thirdly, this application provides an electronic device comprising: a processor and a memory; the memory storing processor-executable instructions; when the processor is configured to execute the instructions, causing the electronic device to implement the method of the first aspect described above.

[0020] Fourthly, this application provides a computer-readable storage medium comprising: computer software instructions; which, when executed in an electronic device, cause the electronic device to implement the method described in the first aspect.

[0021] Fifthly, this application provides a computer program product comprising a computer program; when the computer program is run in an electronic device, it causes the electronic device to implement the method described in the first aspect.

[0022] The beneficial effects of the second to fifth aspects mentioned above are described in the corresponding description of the first aspect and will not be repeated here. Attached Figure Description

[0023] Figure 1 A schematic diagram of the architecture of a system for correcting hallucination assessment data provided in this application embodiment; Figure 2 A flowchart illustrating a method for correcting hallucination assessment data provided in an embodiment of this application; Figure 3 A schematic diagram illustrating a process for correcting hallucination problems provided in an embodiment of this application; Figure 4 A schematic diagram illustrating a correct response correction process provided in an embodiment of this application; Figure 5 A schematic diagram of a hallucination recovery correction process provided for an embodiment of this application; Figure 6 A schematic diagram illustrating a process for correcting hallucination type labels provided in an embodiment of this application; Figure 7 A flowchart illustrating a verification and correction module provided in an embodiment of this application; Figure 8 A flowchart illustrating a method for constructing hallucination assessment data provided in an embodiment of this application; Figure 9 A schematic diagram of a knowledge triple extraction process provided for an embodiment of this application; Figure 10 This application provides a schematic diagram of a process for generating a first question-and-answer pair in an embodiment of the present application. Figure 11 A schematic diagram illustrating the process of generating hallucination responses and hallucination type labels provided in this application embodiment; Figure 12 A flowchart illustrating a generation module provided in an embodiment of this application; Figure 13 A schematic diagram illustrating the composition of a device for correcting hallucination assessment data provided in an embodiment of this application; Figure 14 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0024] The following is a detailed description, with reference to the accompanying drawings, of a method, apparatus, electronic device, and storage medium for correcting hallucination assessment data provided in this application.

[0025] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0026] The terms "first" and "second," etc., used in the specification and drawings of this application are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of objects.

[0027] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0028] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0029] To facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish the same or similar items with essentially the same function and effect. Those skilled in the art can understand that the terms "first" and "second" are not intended to limit the quantity or execution order.

[0030] In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0031] In the field of large language model illusion research, evaluation datasets play a crucial role, and their quality directly affects the accuracy and reliability of the evaluation of model illusion phenomena.

[0032] However, the current evaluation dataset still has some problems. Different annotators have different understandings and judgment criteria for hallucinations, which can lead to the same text being judged differently under different annotations.

[0033] Furthermore, the current solution does not perform rule-based checks on the generated hallucination assessment data, fails to ensure that the generated hallucination assessment data strictly corresponds to the generation rules and types, and does not establish an effective automated review process for the generated hallucination assessment data to ensure that the generated samples strictly follow the established constraints. It also cannot perform targeted calibration and reconstruction of outputs that deviate from the preset hallucination types, which may make the generated hallucination assessment data unreliable and affect the reliability of the hallucination assessment data.

[0034] To address the aforementioned technical problems, this application provides a method for correcting hallucination assessment data. The method involves generating hallucination assessment data from knowledge documents in the target domain, ensuring a high degree of relevance between the data and the specific professional field, as well as the accuracy of its factual basis, thus providing high-quality domain-adapted samples. The validity of the hallucination assessment data is verified using a set of verification models containing multiple verification models with different structures, reducing the risk of missed or false positives due to the limitations of a single model. Furthermore, targeted corrections are made to identified invalid items, enabling iterative optimization of the hallucination assessment data quality. This allows for targeted calibration and reconstruction of hallucination assessment data that does not conform to preset rules and hallucination types, thereby improving the quality and reliability of the hallucination assessment data.

[0035] The embodiments provided in this application will now be described in detail with reference to the accompanying drawings.

[0036] The method for correcting hallucination assessment data provided in this application can be applied to, for example... Figure 1 The system for correcting hallucination assessment data is shown. For example... Figure 1 As shown, the hallucination assessment data correction system 10 of this application includes: electronic device 11.

[0037] In some embodiments, the electronic device 11 can generate hallucination assessment data based on knowledge documents of the target domain, and then use a set of verification models to verify the hallucination assessment data and iteratively correct invalid verification items.

[0038] Optionally, the electronic device 11 can also store data related to the acquired target domain knowledge documents to facilitate data retrieval, processing, and analysis.

[0039] In some embodiments, the electronic device 11 can be a server, such as a single server or a server cluster consisting of multiple servers. In some embodiments, the server cluster can also be a distributed cluster.

[0040] In some embodiments, the electronic device 11 can be a terminal device, such as a mobile phone, tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, cellular phone, personal digital assistant (PDA), augmented reality (AR) / virtual reality (VR) device, etc. This application embodiment does not impose any special limitations on the specific form of the terminal device.

[0041] In some embodiments, the verification model set is used to verify the hallucination assessment data to determine whether the hallucination assessment data conforms to preset generation rules. The verification model set can be constructed by the electronic device 11 or transmitted to the electronic device 11 by other devices.

[0042] For example, labeled samples can be used as training data to train each base model individually. A targeted loss function can be designed to allow for fine-tuning and optimization of the base model, enabling it to achieve the required functionality. The base model can be a large language model that excels in natural language understanding and instruction following.

[0043] In some embodiments, the hallucination assessment data correction system 10 may further include a prompting device 12.

[0044] Specifically, a communication connection is established between the notification device 12 and the electronic device 11. For example, the connection method can be wireless, such as Bluetooth or Wi-Fi; alternatively, it can be wired, such as fiber optic, etc., without limitation. For example, the notification device 12 and the electronic device 11 can be connected to the Internet via a router, thereby establishing the communication connection between them.

[0045] In some embodiments, the prompting device 12 is used to present the hallucination assessment data before and after correction. For example, the prompting device 12 may be a display device, in which case the prompting device 12 presents the hallucination assessment data before and after correction to the user by displaying the hallucination assessment data before and after correction on a display screen.

[0046] It should be noted that the system architecture described in the embodiments of this application is for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and does not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of system architecture, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0047] The following describes a method for correcting hallucination assessment data provided in an embodiment of this application, with reference to the accompanying drawings.

[0048] For example, the method for correcting hallucination assessment data provided in this application can be achieved through... Figure 1 The system for correcting the hallucination assessment data shown is implemented, or, through... Figure 1 The electronic device implementation in the correction system for the hallucination assessment data shown.

[0049] See Figure 2 This is a flowchart illustrating a method for correcting hallucination assessment data provided in an embodiment of this application. Figure 2 As shown, the method for correcting hallucination assessment data provided in this application includes the following steps S201 to S203.

[0050] S201. Generate illusion assessment data for the target domain based on knowledge documents of the target domain.

[0051] The hallucination assessment data includes hallucination problems, correct responses, hallucination responses, and hallucination type labels corresponding to the hallucination responses.

[0052] In some embodiments, a knowledge document can refer to text, charts, or multimedia materials containing organized and valuable information within a specific professional field or discipline. Knowledge documents typically contain verified or recognized concepts, principles, methods, facts, and data within the target domain.

[0053] For example, knowledge documents may include technical standard white papers for a specific field, product specifications, academic research papers, manuals or textbooks written by experts, audited project reports, and regulations or guidelines issued by authoritative organizations.

[0054] In some embodiments, hallucination assessment data is a structured dataset used to train a hallucination detection model. The hallucination assessment data consists of several interrelated data fields, including a hallucinogenic question, a correct response, a hallucinatory response, and a hallucination type label corresponding to the hallucination response. These fields collectively describe a complete assessment instance. The hallucination assessment data is designed to enable the hallucination detection model to identify factual errors in text responses.

[0055] In some embodiments, hallucination questions are questions specifically designed in a controlled environment to proactively induce and assess potential factual errors in a large language model. The purpose of hallucination questions is not to elicit open-ended or creative answers, but rather to construct test cases that identify potentially factual "hallucinations" in the model's responses by comparing them to known standard answers. For example, the design goal of hallucination questions is to expose hallucination deficiencies in the large language model across dimensions such as entities, relations, and logic.

[0056] In some embodiments, a correct response refers to an answer to the hallucination question that is consistent with domain knowledge. The content of a correct response reflects factual information in the knowledge document, and the correct response serves as the benchmark for determining whether a model's response indicates hallucination.

[0057] In some embodiments, hallucination responses are erroneous answers generated based on hallucinogenic questions and inconsistent with factual information in the domain knowledge. Hallucination responses typically conform to linguistic norms and the question's intent on the surface, but their content deviates factually from the original knowledge, contains logical contradictions, or introduces irrelevant information. The goal of generating hallucination responses is to provide negative examples for hallucination detection models, used to train or evaluate the model's ability to identify factual errors.

[0058] For example, hallucinatory responses can include: responses that directly contradict facts in domain knowledge, such as falsifying time, place, values, or relationships; responses that make inferences or additional assertions that cannot be logically derived from given knowledge; responses that are partially correct in form but mixed with inaccurate details; and responses that are completely detached from the knowledge context and generate content irrelevant to the question. The construction of hallucinatory responses must ensure that their errors are not caused by grammatical inconsistencies or irrelevant answers, but rather stem from factual errors.

[0059] In some embodiments, hallucination type labels are data that classifies and identifies the nature of errors contained in hallucination responses. Based on a predefined classification system, hallucination type labels attribute the root causes of inconsistencies between hallucination responses and standard knowledge. Hallucination type labels provide fine-grained supervision for model training, helping to improve the accuracy of hallucination detection models in recognizing specific error patterns.

[0060] In some embodiments, hallucination assessment data for the target domain is generated based on knowledge documents of the target domain. First, at least one knowledge triple is extracted from the knowledge documents of the target domain. The knowledge triple includes a type of domain knowledge in the knowledge document, an entity in the domain knowledge, and an identifier of the target domain. The domain knowledge is factual information in the knowledge document. Then, hallucination assessment data is generated based on the knowledge triple.

[0061] In some embodiments, the identifier of the target domain is data that identifies domain knowledge and the professional categories or disciplines to which its entities belong. The identifier of the target domain provides taxonomic context for knowledge, enabling knowledge to be systematically organized and managed.

[0062] In some embodiments, before generating illusion assessment data based on knowledge documents of the target domain, the knowledge documents can be classified by domain and screened for importance, prioritizing document fragments with high information density and clear factual statements as the starting point for processing, so as to improve the quality of the generated data and the relevance of subsequent model training.

[0063] S202. Validate the hallucination assessment data based on the validation model set.

[0064] The validation model set includes multiple validation models with different structures.

[0065] In some embodiments, the validation model set is a collection of machine learning models with different architectures, training objectives, or knowledge backgrounds. Each member model in the validation model set has the ability to evaluate the factual consistency or logical plausibility of the text. The role of the validation model set is to provide multi-faceted and complementary evaluation perspectives, comprehensively assessing the validity of hallucination evaluation data through independent and potentially divergent analyses of the same hallucination evaluation data.

[0066] For example, the set of verification models may include a deterministic verification model based on rule or knowledge graph matching to judge the factual accuracy of the response based on structured knowledge; a pre-trained language model to evaluate the authenticity of the statement through semantic understanding; and a large language model to directly judge whether the response is a hallucination through instruction-induced judgment.

[0067] In some embodiments, the verification model set includes at least one of the following: a hallucination problem verification model set, a correct response verification model set, a hallucination response verification model set, and a hallucination label verification model set.

[0068] In some embodiments, various verification model sets can be verified through a voting mechanism of the models.

[0069] For example, each validation model independently outputs a validity judgment result. As shown below.

[0070]

[0071] in, Indicates the first The content to be verified for each sample. This indicates the first [number] model in the current category of the validation model set. A validation model, Indicates the first One sample, , This indicates that the verification is invalid. The time indicates that the verification is valid. A "majority voting" strategy is adopted, which integrates the verification results of multiple large language models for the current type of verification model and votes on them. If more than half of the models are deemed valid, the final verification result is [validity]. 1 indicates the verification is valid; otherwise... The value '0' indicates that the verification is invalid. The verification result can be shown below.

[0072]

[0073] in, Indicates the first Validation results of one sample This indicates the number of models in the current type of validation model set.

[0074] In some embodiments, different sets of verification models can implement different verification functions for different content in the hallucination assessment data.

[0075] For example, a hallucinogenic question verification model is used to verify whether a hallucinogenic question meets a preset hallucinogenic question generation rule.

[0076] For example, the correct response verification model is used to verify whether a correct response meets the preset correct response generation rules.

[0077] For example, the hallucination response verification model is used to verify whether the hallucination response meets the preset hallucination response generation rules.

[0078] For example, a hallucination label verification model is used to verify whether the hallucination type label is consistent with the characteristics of the hallucination response.

[0079] S203. If there are invalid items in the hallucination assessment data, the invalid items shall be corrected.

[0080] In some embodiments, invalid items can be corrected based on a correction model; wherein the correction model includes at least one of the following: a hallucination problem correction model, a correct response correction model, a hallucination response correction model, and a hallucination label correction model.

[0081] In some embodiments, the correction model is a type of machine learning model specifically designed to correct specific types of errors in hallucination assessment data. The correction model receives invalid data items, or error diagnosis information corresponding to erroneous items as input, and outputs corrected, compliant data content by understanding the original generation intent and quality rules.

[0082] In some embodiments, when the invalid item is a hallucinogenic problem, the invalid item is corrected based on a knowledge triple.

[0083] For example, if the hallucination problem verification model set does not verify that the hallucination problem meets the preset hallucination problem generation rules, it is necessary to perform hallucination problem correction by inputting the knowledge triple and the hallucination problem to be corrected into the hallucination problem correction model. Alternatively, you can input the reasons for invalid verification provided by the hallucination problem verification model to generate a corrected hallucination problem. The hallucinogenic issue can be corrected as follows:

[0084] in, Represents a knowledge triple. This indicates a hallucinogenic issue that needs to be addressed.

[0085] For example, a hallucination problem correction model can be constructed based on the following: First, the training data consists of a large number of targeted correction samples. Each sample may include a knowledge triple and a hallucinogenic question to be corrected, or it may include a description of the specific rule violation that makes the hallucinogenic question invalid, provided by the validation model (e.g., "the question is not a question, the difficulty is insufficient"), as well as a corrected hallucinogenic question that satisfies all preset rules. These elements are concatenated according to a template to form the input sequence of the model, and the corresponding label is the corrected hallucinogenic question text.

[0086] Then, the constructed training data is input into a base model. The base model can be a large language model that excels in natural language understanding and instruction following, possessing powerful text generation and rewriting capabilities. During training, the model learns the mapping relationship from "the context containing knowledge triples and the hallucinogenic problem to be corrected" to "the corrected hallucinogenic problem." The loss function calculates the difference between each word generated by the model and the labeled word of the corrected hallucinogenic problem. Through supervised fine-tuning, the model not only rewrites sentences but also internalizes the understanding of preset generation rules (such as difficulty and relevance), thus enabling targeted semantic reconstruction and optimization of the problem to be corrected based on specific reasons for ineffectiveness.

[0087] For example, such as Figure 3 As shown, the knowledge triples and the hallucinogenic question are input into the hallucinogenic question verification model for verification. If the verification fails, the knowledge triples and the hallucinogenic question are input into the hallucinogenic question correction model for correction, resulting in the knowledge triples and the corrected hallucinogenic question.

[0088] In some embodiments, invalid items are corrected based on knowledge triples and hallucination issues when the invalid item is a correct response.

[0089] For example, if the correct response verification model set does not meet the preset correct response generation rules, correct response correction needs to be performed, and the knowledge triplet needs to be corrected. Hallucination issues Correct reply to be corrected Correct input response correction model Alternatively, you can input the reason for invalid verification given by the correct response verification model to generate a corrected response. A correct reply can be corrected as follows:

[0090] For example, a correct response correction model can be constructed based on the following: First, the training data consists of a large number of targeted correction samples. Each sample can include knowledge triples, hallucinogenic questions, and correct responses to be corrected. It can also include descriptions of specific rule violations that invalidate correct responses, provided by the validation model (e.g., "factual error: the time '1787' in the response does not match '1687' in the knowledge") or "incomplete answer: the core contribution of the work is not stated"), as well as corrected correct responses that satisfy all preset rules. These elements are concatenated according to a template to form the input sequence of the model, and the corresponding labels are the corrected correct response texts.

[0091] Then, the constructed training data is input into a base model. During training, the model learns the mapping relationship from "context containing knowledge triples, hallucinogenic questions, and the correct response to be corrected" to "corrected correct response". The loss function calculates the difference between each lexical generated by the model and the labeled corrected correct response lexical. Through supervised fine-tuning, targeted semantic reconstruction and optimization of the correct response to be corrected can be performed based on specific reasons for ineffectiveness.

[0092] For example, such as Figure 4 As shown, the knowledge triple, the hallucinogenic question, and the correct response are input into the correct response verification model for verification. If the verification fails, the knowledge triple, the hallucinogenic question, and the correct response are input into the correct response correction model for correction, resulting in the knowledge triple, the hallucinogenic question, and the corrected correct response.

[0093] In some embodiments, when the invalid item is a hallucinatory response, the invalid item is corrected based on the knowledge triple, the hallucinogenic problem, and the correct response.

[0094] For example, if the hallucination response verification model set fails to meet the preset hallucination response generation rules, hallucination response correction needs to be performed, and the knowledge triplet needs to be corrected. Hallucination issues Correct reply And the hallucination response to be corrected Input hallucination response correction model Alternatively, you can input the reason for invalid verification given by the hallucination response verification model to generate a corrected hallucination response. The hallucination recovery correction can be done as follows:

[0095] For example, the hallucination response correction model can be constructed based on the following: First, the training data consists of a large number of targeted correction samples. Each sample can include knowledge triples, hallucinogenic questions, correct responses, and hallucinatory responses to be corrected. It can also include descriptions of specific rule violations that invalidate the hallucinatory responses, provided by the validation model, as well as corrected hallucinatory responses that satisfy all preset rules. These elements are concatenated according to a template to form the input sequence of the model, and the corresponding labels are the corrected hallucinatory response texts.

[0096] Then, the constructed training data is input into a base model. During training, the model learns the mapping relationship from "context containing knowledge triples, hallucinogenic questions, correct responses, and the hallucinatory response to be corrected" to "corrected hallucinatory response". The loss function calculates the difference between each lexical unit generated by the model and the labeled lexical unit of the corrected hallucinatory response. Through supervised fine-tuning, targeted semantic reconstruction and optimization of the hallucinatory response to be corrected can be performed based on the specific reasons for ineffectiveness.

[0097] For example, such as Figure 5 As shown, the knowledge triple, hallucinogenic question, correct response, and hallucinatory response are input into the hallucinatory response verification model for verification. If the verification fails, the knowledge triple, hallucinogenic question, correct response, and hallucinatory response are input into the hallucinatory response correction model for correction, resulting in the knowledge triple, hallucinogenic question, correct response, and corrected hallucinatory response.

[0098] In some embodiments, when the invalid item is a hallucination type label, the invalid item is corrected based on the knowledge triple, hallucination question, correct response, and hallucination response.

[0099] For example, if the hallucination label verification model set verifies that the hallucination type label is an incorrect hallucination type label, label correction needs to be performed to correct the hallucination problem. Correct reply Hallucination recovery With tags to be corrected Input illusion label correction model Alternatively, you can input the reason for invalid verification given by the hallucination label verification model to generate a corrected hallucination type label. The hallucination type label can be corrected as follows:

[0100] For example, the hallucination response correction model can be constructed based on the following: First, the training data consists of a large number of targeted correction samples. Each sample may include a hallucination question, a correct response, a hallucination response, and a hallucination type label to be corrected. It may also include a description of the specific inconsistencies given by the validation model that cause the hallucination type label to be invalid, as well as a corrected hallucination type label that matches the hallucination response characteristics. These elements are concatenated according to a template to form the input sequence of the model, and the corresponding label is the corrected hallucination type label.

[0101] Then, the constructed training data is input into a base model. During training, the model learns the mapping from "context containing hallucinogenic questions, correct responses, hallucinatory responses, and hallucination type labels to be corrected" to "corrected hallucination type labels." The loss function calculates the difference between each word generated by the model and the labeled corrected hallucination type label word. Through supervised fine-tuning, targeted corrections to the hallucination type labels can be made.

[0102] For example, such as Figure 6 As shown, the hallucination question, correct response, hallucination response, and hallucination type label are input into the hallucination label verification model for verification. If the verification fails, the hallucination question, correct response, hallucination response, and hallucination type label are input into the hallucination label correction model for correction, resulting in the hallucination question, correct response, hallucination response, and corrected hallucination type label.

[0103] In some embodiments, the validity of the corrected invalid items can be verified by inputting the correction results into a verification model set for validity verification.

[0104] In some embodiments, if the correction result is verified to be valid, the correction ends; if the correction result is verified to be invalid, the correction is iterated.

[0105] The iterative correction includes iteratively correcting the result of the previous correction.

[0106] In some embodiments, iterative correction refers to a repetitive or incremental correction loop process initiated when a single correction operation fails to make an invalid item pass validity verification. The output of the previous correction operation is used as the input for the current iteration, and the same correction strategy is applied again until a valid result that passes verification is produced, or a preset termination condition is met.

[0107] For example, the preset termination condition can be set to a maximum number of corrections.

[0108] In some embodiments, if the verification fails after reaching the maximum number of corrections, the entire sample, including hallucination questions, correct responses, hallucination responses, and hallucination type labels, is discarded.

[0109] In some embodiments, step S202, which verifies the validity of the hallucination assessment data based on the verification model set, can be implemented as the following steps a1 to a4.

[0110] a1. Use at least a preset number of hallucinogenic question verification models to verify whether the hallucinogenic questions meet the preset hallucinogenic question generation rules.

[0111] In some embodiments, the preset hallucinogenic question generation rule is a predefined text instruction that can guide and constrain the generation of hallucinogenic questions that conform to specific format and content requirements based on a given knowledge triple.

[0112] For example, the pre-defined rules for generating hallucinogenic questions may include requiring the generated questions to be of a certain difficulty, avoiding simple factual restates (such as avoiding closed-ended questions like "Did Newton publish 'Mathematical Principles of Natural Philosophy'?" and prioritizing multi-dimensional questions like "When did Newton publish 'Mathematical Principles of Natural Philosophy' and what are the core contributions of the work?"); the questions must be strongly related to the knowledge content in the knowledge triplet and must not generate irrelevant questions that are divorced from background knowledge, etc.

[0113] In some embodiments, the hallucination question verification model receives a knowledge triple and a hallucination question to be verified as input, and evaluates whether the question meets the preset generation rules through an internal multi-layer attention mechanism and semantic understanding module.

[0114] For example, the hallucination question verification model first parses the knowledge triples, extracting information about core entities and domain knowledge to form an internal factual representation. Next, the verification model analyzes the syntactic structure, interrogative word usage, and semantic focus of the hallucination question to determine if it is a closed-ended question of simple fact restatement. The verification model assesses the semantic correlation between the question's intent and the coverage of the knowledge content, ensuring the question remains within the context of the knowledge. Finally, the verification model outputs a structured verification result, which can include not only a binary judgment of pass or fail but also a quantitative score for the question's difficulty level and relevance, and may generate specific modification suggestions, such as pointing out overly simplistic parts of the question or deviations from the intended knowledge.

[0115] For example, a hallucination verification model can be constructed based on the following: First, the training data consists of a large number of labeled samples. Each sample includes a knowledge triple, a hallucinogenic question, and a rule for generating the hallucinogenic question, which are concatenated according to a specific template. The label corresponding to the sample not only includes the overall judgment of "pass / fail", but can also be refined to specify which rule was violated (e.g., "insufficient difficulty: belongs to a yes / no question"), and can include brief modification suggestions or confidence scores.

[0116] Then, the constructed training data is input into the base model. During training, the model learns the mapping relationship from "input with rule instructions and data" to "structured output", and the loss function calculates the difference between the word units generated by the model and the labeled standard output word units.

[0117] a2. Use at least a preset number of correct response verification models to verify whether the correct responses meet the preset correct response generation rules.

[0118] In some embodiments, the preset correct response generation rule is a predefined text instruction that can guide and constrain the generation of correct responses that meet specific format and content requirements based on a given knowledge triple and a generated hallucinogenic question.

[0119] For example, the preset rules for generating correct responses may include that correct responses must fully and accurately address the question, and be generated strictly based on domain knowledge and entities in the knowledge triple, without introducing external information; correct responses must conform to factual logic, and be unambiguous and consistent (e.g., time, numerical values, and relationship descriptions must be completely consistent with the knowledge content).

[0120] In some embodiments, the correct response verification model receives a knowledge triple, a hallucinogenic question, and a correct response to be verified as input, and evaluates whether the question meets the preset generation rules through an internal multi-layer attention mechanism and semantic understanding module.

[0121] For example, the correct response verification model first parses the knowledge triples, extracting factual information such as entities, attributes, and relationships to form an internal factual baseline. Simultaneously, the model analyzes the hallucinogenic question, clarifying its inquiring intent, the key elements of the required answer, and its logical scope. Next, the verification model analyzes the correct response to be verified, understanding its semantic structure and identifying the stated factual points, logical relationships, and responses to the question. The correct response verification model then performs cross-comparison: on the one hand, it matches each factual assertion in the response with the factual baselines in the knowledge triples, performing item-by-item matching and implication relationship reasoning to ensure the response has not been altered, omitted, or added any additional factual details; on the other hand, it aligns the content of the response with all the key points of the hallucinogenic question, checking whether the response comprehensively and unambiguously addresses the question, and whether there are any irrelevant or evasive answers.

[0122] For example, a correct response validation model can be constructed based on the following: First, the training data consists of a large number of labeled samples. Each sample includes a knowledge triple, a hallucinogenic question, a correct response to be verified, and a pre-defined text for generating correct responses, all concatenated according to a specific instruction template. The labels corresponding to the samples not only include an overall "pass / fail" judgment, but can also be refined to specify which rule was violated (e.g., "factual error: the time in the response does not match the knowledge"), and can include correction suggestions.

[0123] Then, the constructed training data is input into the base model. During training, the model learns the mapping relationship from "input with rule instructions, knowledge, questions, and responses" to "structured validation output." The loss function calculates the difference between each word generated by the model and the labeled standard output word. Through supervised fine-tuning, the model gradually internalizes the semantics of the preset rules and learns how to evaluate whether a correct response meets the preset correct response generation rules.

[0124] a3. Use at least a preset number of hallucination response verification models to verify whether the hallucination responses meet the preset hallucination response generation rules.

[0125] In some embodiments, the preset hallucination response generation rule can be a predefined text instruction that can guide and constrain the generation of seemingly reasonable hallucination responses that conform to a specified error pattern based on knowledge triples, hallucinogenic questions, and correct responses.

[0126] For example, the preset hallucination response generation rules may include: the generated hallucination response must be strongly related to the hallucinogenic question and must not generate content unrelated to the question; the generated hallucination response must be inconsistent with the domain knowledge in the knowledge triplet, that is, it must violate the factual basis; the generated hallucination response must be inconsistent with the correct response, and the difference must correspond to a clear hallucination type; the generated hallucination response only contains the response itself and must not be accompanied by explanatory text such as "this is the wrong answer" or "there is a hallucination here".

[0127] In some embodiments, each hallucination response verification model independently receives complete contextual information, including knowledge triples, hallucinogenic questions, correct responses, preset hallucination types, and the hallucination response to be verified. Each model, based on its internal logic, outputs a structured verification result, which includes at least a judgment on whether each generation rule is met, and an overall conclusion.

[0128] For example, a hallucination response verification model can be constructed based on the following: First, the training data consists of a large number of finely labeled samples. Each sample includes a knowledge triple, a hallucinogenic question, a correct response, a preset hallucination type, a hallucination response to be verified, and the complete text of the preset hallucination response generation rule. All of this information is concatenated according to a designed instruction template, which distinguishes each element. The label corresponding to the sample is a structured output that not only includes a judgment of "overall pass or fail," but also a judgment of compliance with each specific rule (such as question relevance, factual violation, type alignment, and formatting), and can further indicate the specific manifestations of rule violations and suggested directions for correction.

[0129] Then, the constructed training data is input into the base model. During training, the model learns the mapping relationship from "input containing complete context and rule instructions" to "structured validation report". The loss function calculates the difference between each word generated by the model and the labeled standard output word containing multi-dimensional judgments. Through supervised fine-tuning, the model gradually internalizes the semantics of the preset rules and learns how to evaluate whether the hallucination response meets the preset hallucination response generation rules.

[0130] a4. Use at least a preset number of hallucination label verification models to verify whether the hallucination type label and the hallucination response features are consistent.

[0131] In some embodiments, the illusion type label includes at least one of the following: entity error illusion, relation error illusion, attribute error illusion, fact fabrication illusion, instruction inconsistency illusion, context inconsistency illusion, and logical inconsistency illusion.

[0132] For example, entity error illusion is an error in the description of the identity, name, existence, etc. of real-world entities (such as people, places, organizations, concepts, etc.), including entity confusion, entity non-existence, etc.

[0133] For example, the illusion of relational error is a misrepresentation of the relationships between entities (such as causal, subordinate, spatiotemporal, etc.) that contradicts the true logical or factual connections between entities.

[0134] For example, the attribute error illusion is an incorrect description of an entity's characteristics and attributes (such as time, quantity, properties, state, etc.) that does not match the entity's true attributes.

[0135] For example, factual fiction illusions are the generation of facts, events, or concepts that have no basis in reality.

[0136] For example, the illusion of inconsistency in instructions is that the generated content does not follow the user's explicit instructions, deviates from the task objectives or format specifications.

[0137] For example, the context inconsistency illusion occurs when the generated content contradicts the input knowledge triplet, and the content is inconsistent with itself.

[0138] For example, the illusion of logical inconsistency is that there are logical contradictions within the generated content, or that the reasoning process violates basic logical rules (such as reversed cause and effect, self-contradiction, etc.).

[0139] In some embodiments, each hallucination label verification model independently receives complete contextual information, including the hallucination question, correct response, hallucination response, and the hallucination type label to be verified. Each model outputs a structured verification result based on its internal logic, which includes at least a consistency judgment result (such as "consistent" or "inconsistent") and possible reasons (such as "The response error is due to attribute tampering, which does not match the 'entity error' label. It is recommended to change it to 'attribute error'").

[0140] For example, an illusion label verification model can be constructed based on the following: First, the training data consists of a large number of samples. Each sample includes: a hallucination question, a correct response, the hallucination response text, and a hallucination type label to be verified. The label corresponding to the sample is a structured output containing a "consistency judgment" (consistent / inconsistent) and a "reason for inconsistency analysis" (e.g., where the mismatch occurs and what type it should be).

[0141] Then, the constructed training data is input into a base model. During training, the model learns a complex mapping from inputs of "hallucination question, correct response, hallucination response, label" to "consistency analysis." The loss function considers both the accuracy of consistency classification and the quality of causal analysis generation. Through supervised fine-tuning, the model internalizes the discrimination boundaries of various hallucination types and learns to associate text features with abstract type definitions, ultimately verifying whether the hallucination type label and the hallucination response features are consistent.

[0142] In some embodiments, such as Figure 7 As shown, the verification and correction module provided in this application embodiment can correct the generated hallucination assessment data. First, the hallucination problem verification and correction module verifies the hallucination problem, and corrects it if the verification is invalid. Then, the correct response verification and correction module verifies the correct response, and corrects it if the verification is invalid. Next, the hallucination response verification and correction module verifies the hallucination response, and corrects it if the verification is invalid. Finally, the hallucination type label verification and correction module verifies the hallucination type label, and corrects it if the verification is invalid.

[0143] In some embodiments, such as Figure 8As shown, step S201 can be specifically implemented as the following steps S801~S804.

[0144] S801. Based on the knowledge document of the target domain, extract at least one knowledge triple.

[0145] In some embodiments, knowledge documents in the target domain can be obtained by accessing a pre-built knowledge base system, a professional academic database, an authorized document management system, or a designated network resource address. When obtaining knowledge documents, filtering and locating can be performed based on keywords in the target domain, document metadata, or directory structure.

[0146] In some embodiments, domain knowledge refers to authentic information content belonging to a specific professional field or discipline. Domain knowledge reflects objectively existing facts, patterns, or consensus within the domain.

[0147] For example, domain knowledge can include event-specific knowledge describing the occurrence of a specific event, such as discovery, invention, or publication; descriptive knowledge describing the attributes of an entity, such as performance, parameters, and characteristics; and relational knowledge expressing the relationships between entities, such as creation, location, and influence. For instance, the statement "Newton published *Mathematical Principles of Natural Philosophy* in 1687" fully expresses a historical event, including the subject, action, time, and object, and can serve as a direct basis for answering related questions.

[0148] In some embodiments, an entity is an object involved in domain knowledge and serves as the information carrier of knowledge statements. Entities are typically concrete or abstract things that play the role of the subject in the knowledge structure.

[0149] For example, entities can include concrete objects such as people, organizations, and geographical locations with specific names or identifiers; outputs or creations with clear names such as products, works, theories, and regulations; and abstract categories such as methods, principles, concepts, and technical terms. For instance, in the domain knowledge "Newton published Mathematical Principles of Natural Philosophy in 1687," both "Newton" and "Mathematical Principles of Natural Philosophy" are entities that carry historical events.

[0150] In some embodiments, domain tags are data that identifies domain knowledge and the professional categories or disciplines to which its entities belong. Domain tags provide taxonomic context for knowledge, enabling it to be systematically organized and managed.

[0151] For example, domain labels may include first-level domain labels that represent broad academic disciplines, such as "physics", "history", and "computer science"; and second-level domain labels that represent more specific research directions or application scenarios, such as "classical mechanics", "history of British science", and "natural language processing".

[0152] In some embodiments, after acquiring knowledge documents in the target domain, the acquired knowledge documents can be parsed and the text extracted. The document content in non-plain text formats such as PDF and images can be converted into a unified and processable plain text format to prepare standardized input data for the subsequent knowledge triple extraction steps.

[0153] S802. Generate the first question-answer pair based on knowledge triples.

[0154] The first question-and-answer pair includes hallucinogenic questions and correct answers.

[0155] In some embodiments, a knowledge triple can be used as part of the input prompt by calling a large language model generation interface that has been fine-tuned by instructions, and the large language model can be requested to automatically generate a hallucinogenic question and the corresponding correct answer based on the knowledge triple; alternatively, the entities and domain knowledge in the knowledge triple can be automatically filled into the preset question and answer sentence by executing predefined question and answer template filling rules, thereby synthesizing the first question and answer pair.

[0156] In some embodiments, after generating the first question-and-answer pair, the generated first question-and-answer pair can also be checked for logical consistency and factual accuracy. By comparing the correct response with the domain knowledge in the original knowledge triple, it can be ensured that the correct response does not deviate from the original facts and that the hallucinogenic question and the correct response are logically matched.

[0157] S803. Based on the knowledge triplet and the first question-answer pair, generate hallucination responses to hallucination questions, as well as hallucination type tags corresponding to the hallucination responses.

[0158] In some embodiments, based on knowledge triples and a first question-answer pair, a second question-answer generation model can be used to generate hallucinatory responses to hallucination questions, as well as hallucination type labels corresponding to the hallucination responses. S804. Construct hallucination assessment data based on at least one of the following: knowledge triples, first question-answer pair, second question-answer pair, and hallucination type label.

[0159] The second question-and-answer pair includes hallucination questions and hallucination responses, with hallucination assessment data used as sample data to construct a hallucination detection model.

[0160] In some embodiments, the hallucination detection model is a trained machine learning model specifically designed to automatically identify whether a text response contains hallucination content that contradicts or is illogical with a given source of knowledge. The input to the hallucination detection model is typically a question, a response, and optional background knowledge; the output is a judgment on whether the response contains hallucinations, or further, a classification of the hallucination type.

[0161] In some embodiments, standardized hallucination evaluation data can be obtained by automatically assembling and serializing all intermediate data products generated in the preceding steps, including knowledge triples, first question-answer pairs, second question-answer pairs, and hallucination type labels, according to a preset data structure template. The assembled data can also be saved as a file or written to a database for direct reading in subsequent model training processes.

[0162] In some embodiments, after constructing the hallucination assessment data, the assembled hallucination assessment dataset can be cleaned and deduplicated to remove samples containing invalid formats, null values, or duplicate content. Statistical analysis can be performed on the distribution of different hallucination types in the dataset, and if necessary, the class balance can be adjusted using oversampling or undersampling techniques to construct a higher quality sample dataset.

[0163] In some embodiments, step S801 can be implemented as the following steps.

[0164] In some embodiments, triples can be used to extract knowledge documents of the target domain from a large language model, extracting knowledge triples. The extracted knowledge triples can be divided into multiple categories, where domain knowledge, entities, and domain labels of the same category are grouped into one knowledge triple.

[0165] For example, domain knowledge and core entity extraction are performed based on a large language model extracted from triples. The input is knowledge documents in the target domain (such as authoritative knowledge bases, technical documents, books, and web pages in fields such as medicine, finance, and technology; common document formats include TXT, JSON / JSONL, and CSV). Let the document set be denoted as . ,in This represents the number of documents.

[0166] Next, large language models are extracted based on triples. It performs semantic parsing and information extraction on knowledge documents, identifying domain knowledge, entities (such as people, places, organizations, concepts, parameters, etc.) and their respective domains in the documents. It is possible to extract knowledge and entities from multiple domains from the same document. ,in, .in It is the first Domain knowledge of each sample It is the first The entities of each sample, It is the first Domain labels for each sample.

[0167] Ultimately, through the study of Domain knowledge and entity extraction are performed to form a set of structured knowledge triples. .

[0168] For example, a large language model extracted from triples can be constructed based on the following: First, the training process involves feeding the model a large number of document paragraph samples from the target domain and their corresponding labeled knowledge triple examples. This allows the model to learn and master the rules for mapping natural language descriptions to a standard "(domain knowledge, core entity, domain label)" format. For each training sample, the label is the corresponding knowledge triple, generated manually or automatically.

[0169] Secondly, the optimization objective of the model can be defined as a triplet extraction loss function, which takes into account the accuracy of the reconstruction of domain knowledge text, the correctness of the boundary recognition of core entities, and the classification accuracy of domain labels.

[0170] Then, in terms of output design, the model is trained to generate the three components in a formatted manner according to preset special characters (such as specific delimiters or tags), or directly output a structured JSON object.

[0171] Ultimately, the completed triplet extraction large language model can analyze the semantics of text and extract structured knowledge units that meet the requirements when receiving new knowledge document input.

[0172] For example, such as Figure 9 As shown, knowledge documents collected from domain knowledge bases, technical documents, books, or web pages are input into the triple extraction large language model to finally obtain knowledge triples. A knowledge triple includes domain knowledge, an entity, and the identifier of the target domain.

[0173] In some embodiments, step S802 can be implemented as step b1 or b2.

[0174] b1. Based on knowledge triples, the first question-and-answer pair can be generated by inputting the knowledge triples and the first prompt words into the first question-and-answer generation model. The first question-and-answer generation model uses the knowledge triples as contextual information and combines them with the first prompt words to generate the first question-and-answer pair.

[0175] The first cue word is used to induce the generation of hallucination questions and correct responses.

[0176] For example, knowledge triples As contextual information, it is input together with the first prompt word into the first question-answering generation model. Generate the first question-and-answer pair, namely: ,in, Referring to hallucination issues, A correct reply. The first prompt word.

[0177] In some embodiments, the first prompt word is a predefined text instruction. The first prompt word can guide and constrain the behavior of the first question-answer generation model, enabling the model to generate question-answer pairs that conform to specific format and content requirements based on given knowledge triples.

[0178] For example, the first prompt can clearly define the rules for generating the first question and answer pair, ensuring the difficulty of the hallucinogenic question and the accuracy of the correct response. The first prompt can include: the generated question must be of a certain difficulty, avoiding simple factual restatements (such as avoiding closed-ended questions like "Did Newton publish *Mathematical Principles of Natural Philosophy*?" and prioritizing multi-dimensional questions like "When did Newton publish *Mathematical Principles of Natural Philosophy* and what are the core contributions of this work?"); the question must be strongly related to the knowledge content in the knowledge triple and must not generate irrelevant questions detached from background knowledge; the correct response must completely and accurately address the hallucinogenic question and be strictly based on the domain knowledge and entities in the knowledge triple, without introducing external information; the correct response must conform to factual logic, without ambiguity or contradiction (such as time, numerical values, and relational descriptions must be completely consistent with the knowledge content).

[0179] In some embodiments, the first question-answer generation model refers to a large language model that has been fine-tuned with instructions and is specifically designed to generate hallucinogenic questions and correct responses. The role of the first question-answer generation model is to transform structured knowledge triples into first question-answer pairs in natural language form.

[0180] For example, the first question-answer generation model can be constructed based on the following: First, the training data for the first question-answering generation model consists of input-output paired samples. The input samples are composed of two parts: a designed prompt word template and specific knowledge triples, filled into the template in a structured or natural language description. The tags are corresponding, manually written or verified question-answer pairs, i.e., a natural language question and its answer targeting domain knowledge.

[0181] Secondly, a pre-trained language model can be chosen as the base, such as a model based on a Decoder-only architecture or an Encoder-Decoder architecture. The model's input format is designed to simultaneously accept prompt words and knowledge triples, and output question-answer pairs.

[0182] Finally, supervised fine-tuning is used to train the pedestal model. During training, the model learns the mapping from "cue word + knowledge triple" to "complete question-answer pair". The loss function is typically a standard next-word prediction loss (such as cross-entropy loss), and the model optimizes by minimizing the difference between the generated question-answer pair text and the labeled text in the training data.

[0183] For example, such as Figure 10 As shown, the knowledge triples and the first prompt word are input into the first question-and-answer generation model, which will eventually output the first question-and-answer pair. The first question-and-answer pair includes the hallucinogenic question and the correct answer to the hallucinogenic question.

[0184] b2. Based on knowledge triples, generate the first question-and-answer pair. Alternatively, by executing predefined question-and-answer template filling rules, the entities and domain knowledge in the knowledge triples can be automatically filled into the preset question and response sentences to synthesize the first question-and-answer pair.

[0185] For example, firstly, based on the characteristics of the knowledge triples, a matching predefined question-and-answer template is selected from the template library. Then, according to rules, entity and domain knowledge is extracted and parsed from the knowledge triples and filled into placeholders in the question sentence to form a complete hallucinogenic question; simultaneously, the parsed entity or domain knowledge text is filled into placeholders in the response sentence to form a complete correct response. Finally, the generated question and response are combined and output as the first question-and-answer pair.

[0186] In some embodiments, the question phrase is a fixed sentence structure within a predefined question-and-answer template specifically designed to construct the hallucinogenic question section. The question phrase has built-in specific grammar and interrogative words, and includes positional markers for embedding specific content of knowledge triples.

[0187] For example, question formats may include: attribute-based questions with the structure "What is [a certain attribute] of [entity]?"; relationship-based questions with the structure "What is the relationship between [entity] and [entity]?"; or open-ended questions with the structure "Please describe the role of [entity] in [an event]". [] represents placeholders.

[0188] In some embodiments, the response phrase is a fixed sentence structure within a predefined question-and-answer template specifically designed to construct the correct response. The response phrase provides a standard way of stating facts and includes position markers for filling in specific factual content derived from knowledge triples that corresponds to the question.

[0189] For example, response formats may include: attribute statement formats with the structure "[entity]'s [a certain attribute] is [attribute content]"; relation statement formats with the structure "[entity] and [entity] have a [relationship]", etc. Here, [] represents placeholders.

[0190] In some embodiments, the fill rules are a logically defined set of instructions that define how to map and replace the different components of a knowledge triple into corresponding placeholders in a predefined question-answer template. The fill rules specify the transformation logic from structured knowledge to natural language sentences, ensuring that the final synthesized question-answer pairs are semantically consistent with the original knowledge.

[0191] For example, the filling rules may include: a rule that "replaces the [entity] placeholder in the question and response statements with the entity field value in the knowledge triple"; a rule that "replaces the [attribute content] placeholder in the response statement with the domain knowledge field value in the knowledge triple, either as a whole or after partial parsing"; and conditional logic rules that may exist to select different question-and-answer template variants based on the knowledge type.

[0192] In some embodiments, step S803 can be implemented as the following steps.

[0193] In some embodiments, a knowledge triple, a first question-and-answer pair, a preset hallucination type, and a second prompt word can be input into a second question-and-answer generation model. The second question-and-answer generation model uses the knowledge triple as contextual information and combines it with the first question-and-answer pair, the preset hallucination type, and the second prompt word to generate a hallucination response to the hallucination question and a hallucination type label corresponding to the hallucination response. The second prompt word is used to induce the generation of a hallucination response to the hallucination question, and the hallucination response satisfies the preset hallucination type.

[0194] For example, knowledge triples As contextual information, related to hallucination issues Correct reply Hallucination response prompts and preset hallucination types Common Input Second Question Answering Generation Model This generates a corresponding hallucinatory response. And corresponding hallucination response tags ,Right now:

[0195] In some embodiments, a preset hallucination type is a predefined and enumerated set of standard categories used to classify and describe the causes of factual errors. The preset hallucination type provides explicit error direction constraints and classification criteria for the generation of hallucination responses, ensuring that the generated error samples can systematically cover hallucination phenomena of different natures.

[0196] In some embodiments, the second prompt is a predefined text instruction. The second prompt can guide and constrain the behavior of the second question-answering generation model, enabling the model to generate seemingly reasonable hallucinatory responses that conform to a specified error pattern, based on a full understanding of the knowledge triples, hallucinogenic questions, and correct responses.

[0197] For example, the second prompt word could be a clear rule for generating hallucination responses to ensure the validity and accuracy of the labeling, including: the generated hallucination response must be strongly related to the hallucinogenic question and must not generate content unrelated to the question; the generated hallucination response must be inconsistent with the domain knowledge in the knowledge triplet, i.e., it must violate the factual basis; the generated hallucination response must be inconsistent with the correct response, and the difference must correspond to a clear type of hallucination; the generated hallucination response only contains the response itself and must not be accompanied by explanatory text such as "this is the wrong answer" or "hallucination exists here".

[0198] In some embodiments, the second question-answering generation model refers to a large language model that has been fine-tuned with instructions specifically for generating hallucination responses. The role of the second question-answering generation model is to synthesize a hallucination response that meets the preset hallucination type requirements based on the prompting of the second prompt word and in combination with all the contextual information of the input, and to correctly classify it.

[0199] For example, the second question-answering generation model can be constructed based on the following: First, the input samples for the model's training data include knowledge triples, a first question-answer pair (containing a hallucinogenic question and a correct response), a preset hallucination type, and a second cue word. The labels are the corresponding, manually written or verified hallucination responses and the hallucination type labels matched to those responses.

[0200] Then, the structure and training method of the second question-answer generation model can be set up with reference to the first question-answer generation model.

[0201] For example, such as Figure 11 As shown, the knowledge triple, the first question-and-answer pair, the second prompt word, and the preset hallucination type are input into the second question-and-answer generation model, and the final output is the hallucination response and the hallucination type label.

[0202] In some embodiments, such as Figure 12 As shown in the illustration, the generation module provided in this application can generate the required hallucination assessment data. First, through the domain knowledge and entity extraction module, knowledge triples can be generated from the knowledge document. Next, through the hallucination question and correct response generation module, a first question-and-answer pair can be generated based on the knowledge triples. Finally, through the hallucination question and hallucination response generation module, hallucination responses and hallucination type tags can be generated based on the knowledge triples and the first question-and-answer pair, which together constitute the hallucination assessment data.

[0203] As can be seen, the above mainly describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, the embodiments of this application provide corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the modules and algorithm steps of the various examples described in the embodiments disclosed herein, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention.

[0204] This application embodiment can divide the hallucination assessment data correction device into functional modules according to the above method example. For example, each function can be divided into a separate functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. Optionally, the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0205] In some embodiments, this application also provides a device for correcting hallucination assessment data. This device may include one or more functional modules for implementing the hallucination assessment data correction method described in the above method embodiments.

[0206] For example, Figure 13 This is a schematic diagram illustrating the composition of a hallucination assessment data correction device 1300 provided in an embodiment of this application. Figure 13 As shown, the device for correcting the hallucination assessment data includes a generation module 1301 and a processing module 1302.

[0207] The generation module 1301 is used to generate hallucination assessment data for the target domain based on knowledge documents of the target domain; the processing module 1302 is used to verify the validity of the hallucination assessment data based on a verification model set; the verification model set includes multiple verification models with different structures; and the invalid items are corrected when there are invalid items in the hallucination assessment data.

[0208] In some embodiments, the verification model set includes at least one of the following: a hallucination question verification model set, a correct response verification model set, a hallucination response verification model set, and a hallucination label verification model set; the hallucination question verification model is used to verify whether the hallucination question meets the preset hallucination question generation rules; the correct response verification model is used to verify whether the correct response meets the preset correct response generation rules; the hallucination response verification model is used to verify whether the hallucination response meets the preset hallucination response generation rules; and the hallucination label verification model is used to verify whether the hallucination type label and the hallucination response have consistent characteristics.

[0209] In some other embodiments, the processing module 1302 is further configured to correct invalid items based on a correction model; wherein the correction model includes at least one of the following: a hallucination problem correction model, a correct response correction model, a hallucination response correction model, and a hallucination label correction model.

[0210] In some other embodiments, the generation module 1301 is further configured to extract at least one knowledge triple based on a knowledge document of the target domain, the knowledge triple including a type of domain knowledge in the knowledge document, an entity in the domain knowledge, and an identifier of the target domain, wherein the domain knowledge is factual information in the knowledge document; and generate hallucination assessment data based on the knowledge triple.

[0211] In some other embodiments, the processing module 1302 is further configured to: correct the invalid item based on a knowledge triple when the invalid item is a hallucinogenic question; correct the invalid item based on a knowledge triple and a hallucinogenic question when the invalid item is a correct response; correct the invalid item based on a knowledge triple, a hallucinogenic question, and a correct response when the invalid item is a hallucination response; and correct the invalid item based on a knowledge triple, a hallucinogenic question, a correct response, and a hallucination response when the invalid item is a hallucination type label.

[0212] In some other embodiments, the processing module 1302 is further configured to verify the validity of the correction result; if the correction result is verified to be valid, the correction is terminated; if the correction result is verified to be invalid, the correction is iteratively performed; wherein the iterative correction includes iteratively correcting the previous correction result.

[0213] In the case of implementing the functions of the integrated modules described above in hardware, this embodiment of the invention provides a possible structural schematic diagram of the electronic device involved in the above embodiments. For example... Figure 14 As shown, the electronic device 900 includes: a processor 902, a communication interface 903, and a bus 904. Optionally, the electronic device 900 may also include a memory 901.

[0214] Processor 902 may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 902 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 902 may also be a combination that implements computing functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0215] The communication interface 903 is used to connect to other devices via a communication network. This communication network can be Ethernet, wireless access network, wireless local area network (WLAN), etc.

[0216] The memory 901 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0217] In one possible implementation, the memory 901 can exist independently of the processor 902. The memory 901 can be connected to the processor 902 via a bus 904 and is used to store instructions or program code. When the processor 902 calls and executes the instructions or program code stored in the memory 901, it can implement the method for correcting hallucination evaluation data provided in this embodiment of the invention.

[0218] In another possible implementation, the memory 901 can also be integrated with the processor 902.

[0219] The 904 bus can be an extended industry standard architecture (EISA) bus, etc. The 904 bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 14 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0220] Through the above description of the implementation methods, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the service calling device can be divided into different functional modules to complete all or part of the functions described above.

[0221] This application also provides a computer-readable storage medium. All or part of the processes in the above method embodiments can be executed by computer instructions instructing related hardware. The program can be stored in the aforementioned computer-readable storage medium, and when executed, it can include the processes of the above method embodiments. The computer-readable storage medium can be any of the foregoing embodiments or memory. The aforementioned computer-readable storage medium can also be an external storage device of the aforementioned service invocation device, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the aforementioned service invocation device. Further, the aforementioned computer-readable storage medium can include both internal storage units of the aforementioned service invocation device and external storage devices. The aforementioned computer-readable storage medium is used to store the aforementioned computer program and other programs and data required by the aforementioned service invocation device. The aforementioned computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0222] This application also provides a computer program product comprising a computer program that, when run on a computer, causes the computer to execute any of the illusion assessment data correction methods provided in the above embodiments.

[0223] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for correcting hallucination assessment data, characterized in that, The method includes: Hallucination assessment data for the target domain is generated based on knowledge documents of the target domain; the hallucination assessment data includes hallucinogenic questions, correct responses, hallucination responses, and hallucination type tags corresponding to the hallucination responses; The validity of the hallucination assessment data is verified based on a set of verification models; the set of verification models includes multiple verification models with different structures. If invalid entries exist in the hallucination assessment data, the invalid entries are corrected.

2. The method according to claim 1, characterized in that, The verification model set includes at least one of the following: hallucination problem verification model set, correct response verification model set, hallucination response verification model set, and hallucination label verification model set; The hallucination problem verification model is used to verify whether the hallucination problem satisfies the preset hallucination problem generation rules; The correct response verification model is used to verify whether the correct response meets the preset correct response generation rules; The hallucination response verification model is used to verify whether the hallucination response meets the preset hallucination response generation rules; The hallucination label verification model is used to verify whether the hallucination type label is consistent with the characteristics of the hallucination response.

3. The method according to claim 1, characterized in that, The correction of the invalid item includes: The invalid items are corrected based on the correction model; wherein the correction model includes at least one of the following: hallucination problem correction model, correct response correction model, hallucination response correction model, and hallucination label correction model.

4. The method according to claim 1, characterized in that, The generation of hallucination assessment data for the target domain based on knowledge documents includes: Based on the knowledge document of the target domain, at least one knowledge triple is extracted. The knowledge triple includes a type of domain knowledge in the knowledge document, an entity in the domain knowledge, and an identifier of the target domain. The domain knowledge is factual information in the knowledge document. The hallucination assessment data is generated based on the knowledge triples.

5. The method according to claim 4, characterized in that, The step of correcting invalid entries in the hallucination assessment data includes: In the case where the invalid item is a hallucinogenic problem, the invalid item is corrected based on the knowledge triple; If the invalid item is a correct response, the invalid item is corrected based on the knowledge triple and the hallucination problem; In the case where the invalid item is a hallucination response, the invalid item is corrected based on the knowledge triplet, the hallucination problem, and the correct response; In the case where the invalid item is a hallucination type label, the invalid item is corrected based on the knowledge triple, the hallucination question, the correct response, and the hallucination response.

6. The method according to claim 1, characterized in that, The correction of the invalid item includes: The validity of the correction results is verified; If the correction result is verified to be valid, the correction process ends. If the correction result is found to be invalid, iterative correction is performed; wherein, the iterative correction includes iterative correction of the previous correction result.

7. A device for correcting hallucination assessment data, characterized in that, include: Generation module, processing module; The generation module is used to generate illusion assessment data for the target domain based on knowledge documents of the target domain; The processing module is used to validate the hallucination assessment data based on the validation model set; The verification model set includes multiple verification models with different structures; If invalid entries exist in the hallucination assessment data, the invalid entries are corrected.

8. An electronic device, characterized in that, The device includes a processor and a memory, the processor being coupled to the memory; the memory is used to store computer instructions, which are loaded and executed by the processor to enable the computer device to implement the method for correcting hallucination assessment data as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer-executable instructions that, when executed on a computer, cause the computer to perform the method for correcting hallucination assessment data according to any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when run on an electronic device, causes the electronic device to perform a method for correcting hallucination assessment data as described in any one of claims 1 to 6.

Images