Construction method, device, storage medium and program product of hallucination evaluation data

Abstract

This application provides a method, apparatus, storage medium, and program product for constructing hallucination assessment data, relating to the field of computer technology. It aims to increase the coverage of hallucination assessment data, clarify the classification of hallucination types, and improve the quality of the hallucination assessment data. The method includes: extracting at least one knowledge triple based on a knowledge document of a target domain; generating a first question-and-answer pair based on the knowledge triple; generating a hallucination response to a hallucination question and a hallucination type label corresponding to the hallucination response based on the knowledge triple and the first question-and-answer pair; and constructing hallucination assessment data based on at least one of the knowledge triple, the first question-and-answer pair, a second question-and-answer pair, and the hallucination type label.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to methods, devices, storage media, and program products for constructing hallucination assessment data. Background Technology

[0002] With the widespread application of large language models in natural language processing, question answering, and content generation, their "illusion phenomenon" has become a key issue restricting the reliability of these models. The illusion phenomenon of large language models refers to the generation of content that is inconsistent with reality, logically contradictory, or deviates from the context. In the field of research on large language model illusions, illusion evaluation datasets play a crucial role, and their quality directly affects the accuracy and reliability of evaluating model illusion phenomena.

[0003] Current methods for generating hallucination assessment data primarily rely on builders with domain-specific knowledge designing questions based on established knowledge sources and writing corresponding responses that contain factual errors.

[0004] However, existing hallucination assessment data mostly rely on manual annotation, which leads to problems such as high annotation costs, small scale, limited coverage of scenarios, and unclear classification of hallucination types, making high-quality hallucination assessment data scarce. Summary of the Invention

[0005] This application provides a method, device, storage medium, and program product for constructing hallucination assessment data, which is used to increase the coverage of hallucination assessment data, make the classification of hallucination types clearer, and improve the quality of hallucination assessment data.

[0006] In a first aspect, this application provides a method for constructing hallucination assessment data, comprising: extracting at least one knowledge triple based on a knowledge document of a target domain, wherein the knowledge triple includes a type of domain knowledge in the knowledge document, an entity in the domain knowledge, and an identifier of the target domain, and the domain knowledge is factual information in the knowledge document; generating a first question-and-answer pair based on the knowledge triple, wherein the first question-and-answer pair includes a hallucinogenic question and a correct response; generating a hallucinogenic response to the hallucinogenic question and a hallucination type label corresponding to the hallucination response based on the knowledge triple and the first question-and-answer pair; constructing hallucination assessment data based on at least one of the knowledge triple, the first question-and-answer pair, the second question-and-answer pair, and the hallucination type label, wherein the second question-and-answer pair includes a hallucinogenic question and a hallucination response, and the hallucination assessment data is used as sample data for constructing a hallucination detection model or for evaluating the severity of hallucination phenomena generated by a large language model.

[0007] The technical solution provided in this application offers at least the following benefits: By extracting knowledge triples from knowledge documents in the target domain, the accuracy, structure, and domain specificity of the information upon which subsequent processing is based are ensured. This allows the constructed hallucination assessment data to be optimized for the knowledge characteristics of the target domain. Consequently, only the knowledge documents need to be modified to generate hallucination assessment data for the target domain, increasing the coverage scenarios of the hallucination assessment data. Based on this triple, a first question-and-answer pair containing hallucinogenic questions and correct responses is generated, establishing a real and reliable question-and-answer benchmark for the entire hallucination assessment data construction process. Furthermore, by combining this knowledge triple with the first question-and-answer pair, hallucination responses and their corresponding hallucination type labels are generated, forming a second question-and-answer pair. This enables a clearer classification of hallucination types in the hallucination assessment data, avoiding disputes in the classification of hallucination assessment data. Moreover, this method, which does not rely on manual annotation, can significantly reduce costs and increase data scale. Ultimately, hallucination assessment data is constructed based on at least one of the following: knowledge triples, first question-and-answer pairs, second question-and-answer pairs, and hallucination type labels. This ensures that the hallucination assessment data includes complete hallucination questions, hallucination responses, hallucination types, as well as knowledge triples containing the correct answers and data sources, thereby improving the quality of the generated hallucination assessment data.

[0008] One possible implementation involves inputting a knowledge triple and a first prompt word into a first question-and-answer generation model. The first question-and-answer generation model uses the knowledge triple as contextual information and combines it with the first prompt word to generate a first question-and-answer pair. The first prompt word is used to induce the generation of illusion questions and correct responses.

[0009] Another possible implementation involves inputting one type of domain knowledge and entity from the knowledge triple into a preset question and response format to obtain the first question-answer pair.

[0010] Another possible implementation involves inputting a knowledge triple, a first question-and-answer pair, a preset hallucination type, and a second prompt word into a second question-and-answer generation model. The second question-and-answer generation model uses the knowledge triple as contextual information and combines it with the first question-and-answer pair, the preset hallucination type, and the second prompt word to generate a hallucination response to the hallucination question, as well as a hallucination type label corresponding to the hallucination response. The second prompt word is used to induce the generation of a hallucination response to the hallucination question, and the hallucination response satisfies the preset hallucination type.

[0011] Another possible implementation is that the preset illusion type includes at least one of the following: entity error illusion, relation error illusion, attribute error illusion, fact fabrication illusion, instruction inconsistency illusion, context inconsistency illusion, and logic inconsistency illusion.

[0012] Another possible approach is to validate the hallucination assessment data and correct any invalid entries found in the data.

[0013] Secondly, this application provides an apparatus for constructing hallucination assessment data, comprising: an extraction module and a processing module; the extraction module is used to extract at least one knowledge triple based on knowledge documents in a target domain; the processing module is used to generate a first question-and-answer pair based on the knowledge triple; generate a second question-and-answer pair based on the knowledge triple and the first question-and-answer pair; and construct sample data for a hallucination annotation model based on the second question-and-answer pair and the tags corresponding to the hallucination responses.

[0014] One possible implementation is that the processing module is also used to input the knowledge triples and the first prompt word into the first question-answer generation model. The first question-answer generation model uses the knowledge triples as contextual information and combines them with the first prompt word to generate the first question-answer pair. The first prompt word is used to induce the generation of illusion questions and correct responses.

[0015] Another possible implementation is that the processing module is also used to input a type of domain knowledge and entity from the knowledge triple into a preset question and response sentence to obtain the first question-answer pair.

[0016] Another possible implementation involves a processing module that inputs a knowledge triple, a first question-and-answer pair, a preset hallucination type, and a second prompt word into a second question-and-answer generation model. The second question-and-answer generation model uses the knowledge triple as contextual information and combines it with the first question-and-answer pair, the preset hallucination type, and the second prompt word to generate a hallucination response to the hallucination question, as well as a hallucination type label corresponding to the hallucination response. The second prompt word is used to induce the generation of a hallucination response to the hallucination question, and the hallucination response satisfies the preset hallucination type.

[0017] Another possible implementation is that the preset illusion type includes at least one of the following: entity error illusion, relation error illusion, attribute error illusion, fact fabrication illusion, instruction inconsistency illusion, context inconsistency illusion, and logic inconsistency illusion.

[0018] Another possible implementation is that the processing module is also used to validate the hallucination assessment data; if invalid items exist in the hallucination assessment data, the invalid items are corrected.

[0019] Thirdly, this application provides an electronic device comprising: a processor and a memory; the memory storing processor-executable instructions; when the processor is configured to execute the instructions, causing the electronic device to implement the method of the first aspect described above.

[0020] Fourthly, this application provides a computer-readable storage medium comprising: computer software instructions; which, when executed in an electronic device, cause the electronic device to implement the method described in the first aspect.

[0021] Fifthly, this application provides a computer program product comprising a computer program; when the computer program is run in an electronic device, it causes the electronic device to implement the method described in the first aspect.

[0022] The beneficial effects of the second to fifth aspects mentioned above are described in the corresponding description of the first aspect and will not be repeated here. Attached Figure Description

[0023] Figure 1 A schematic diagram of the architecture of a system for constructing hallucination assessment data provided in this application embodiment; Figure 2 A flowchart illustrating a method for constructing hallucination assessment data provided in an embodiment of this application; Figure 3 A schematic diagram of a knowledge triple extraction process provided for an embodiment of this application; Figure 4 This application provides a schematic diagram of a process for generating a first question-and-answer pair in an embodiment of the present application. Figure 5 A schematic diagram illustrating the process of generating hallucination responses and hallucination type labels provided in this application embodiment; Figure 6 A flowchart illustrating a generation module provided in an embodiment of this application; Figure 7 A flowchart illustrating a method for correcting hallucination assessment data provided in an embodiment of this application; Figure 8 A schematic diagram illustrating a process for correcting hallucination problems provided in an embodiment of this application; Figure 9 A schematic diagram illustrating a correct response correction process provided in an embodiment of this application; Figure 10 A schematic diagram of a hallucination recovery correction process provided for an embodiment of this application; Figure 11 A schematic diagram illustrating a process for correcting hallucination type labels provided in an embodiment of this application; Figure 12 A flowchart illustrating a verification and correction module provided in an embodiment of this application; Figure 13 A schematic diagram illustrating the composition of a device for constructing hallucination assessment data, provided in an embodiment of this application; Figure 14 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0024] The following is a detailed description, with reference to the accompanying drawings, of a method, apparatus, storage medium, and program product for constructing hallucination assessment data provided in this application.

[0025] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0026] The terms "first" and "second," etc., used in the specification and drawings of this application are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of objects.

[0027] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0028] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0029] To facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish the same or similar items with essentially the same function and effect. Those skilled in the art can understand that the terms "first" and "second" are not intended to limit the quantity or execution order.

[0030] In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0031] With the widespread application of large language models in natural language processing, question answering, and content generation, their "illusion phenomenon" has become a key issue restricting the reliability of these models. The illusion phenomenon of large language models refers to the generation of content that is inconsistent with reality, logically contradictory, or deviates from the context. In the field of research on large language model illusions, illusion evaluation datasets play a crucial role, and their quality directly affects the accuracy and reliability of evaluating model illusion phenomena.

[0032] Current methods for generating hallucination assessment data primarily rely on builders with domain-specific knowledge designing questions based on established knowledge sources and writing corresponding responses containing factual errors. Differences in the understanding and judgment criteria of hallucinations among different annotators lead to the same text being judged differently under different annotations. For some semantically ambiguous or controversial text content, some annotators consider it a model hallucination, while others consider it a reasonable inference. This inconsistency in annotation severely affects the accuracy of dataset annotation, thus interfering with the evaluation of model hallucinations. With the widespread application of large language models in various industries, new task requirements are constantly emerging, such as the interpretation and application of complex legal clauses in intelligent legal consultation, and the generation of personalized learning path planning suggestions for students in intelligent education. Existing assessment datasets have not been optimized for the characteristics of these new tasks, making it difficult to evaluate the model's hallucination performance when performing new tasks. Therefore, current methods for generating hallucination assessment data still suffer from problems such as inflexible coverage of scenarios and inconsistent annotation.

[0033] In summary, existing hallucination assessment data largely relies on manual annotation, which leads to problems such as high annotation costs, small scale, limited coverage of scenarios, and unclear classification of hallucination types, resulting in a scarcity of high-quality hallucination assessment data.

[0034] To address the aforementioned technical problems, this application provides a method for constructing hallucination assessment data. The method involves extracting knowledge triples from knowledge documents in the target domain, ensuring the accuracy, structure, and domain specificity of the information used in subsequent processing. This allows the constructed hallucination assessment data to be optimized for the knowledge characteristics of the target domain. Therefore, only modifications to the knowledge documents are needed to generate hallucination assessment data for the target domain, increasing the coverage scenarios of the hallucination assessment data. Based on these triples, a first question-and-answer pair containing hallucinogenic questions and correct responses is generated, establishing a realistic and reliable question-and-answer benchmark for the entire hallucination assessment data construction process. Furthermore, combining the knowledge triples with the first question-and-answer pair, hallucination responses and their corresponding hallucination type labels are generated, forming a second question-and-answer pair. This allows for a clearer classification of hallucination types in the hallucination assessment data, avoiding disputes in the classification of hallucination assessment data. Moreover, this method, which does not rely on manual annotation, significantly reduces costs and increases data scale. Ultimately, hallucination assessment data is constructed based on at least one of the following: knowledge triples, first question-and-answer pairs, second question-and-answer pairs, and hallucination type labels. This ensures that the hallucination assessment data includes not only complete hallucination questions, hallucination responses, and hallucination types, but also knowledge triples containing the correct answers and data sources, thereby improving the quality of the generated hallucination assessment data.

[0035] The embodiments provided in this application will now be described in detail with reference to the accompanying drawings.

[0036] The method for constructing hallucination assessment data provided in this application can be applied to, for example... Figure 1 The system for constructing hallucination assessment data is shown. For example... Figure 1 As shown, the hallucination assessment data construction system 10 of this application includes: electronic device 11.

[0037] In some implementations, the electronic device 11 can generate data such as hallucinogenic problems, correct responses, hallucinatory responses to hallucinogenic problems, and hallucinatory type tags corresponding to hallucinatory responses based on knowledge documents of the target domain, and further construct hallucinatory assessment data based on the generated data.

[0038] Optionally, the electronic device 11 can also store knowledge documents related to the target domain to facilitate data retrieval, processing, and analysis.

[0039] In some embodiments, electronic device 11 can be a server, for example, a single server, or a server cluster consisting of multiple servers. In some embodiments, the server cluster can also be a distributed cluster.

[0040] In some embodiments, the electronic device 11 can be a terminal device, such as a mobile phone, tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, cellular phone, personal digital assistant (PDA), augmented reality (AR) / virtual reality (VR) device, etc. This application embodiment does not impose any special limitations on the specific form of the terminal device.

[0041] In some implementations, the hallucination assessment data construction system 10 may also include a prompting device 12.

[0042] A communication connection is established between electronic device 11 and notification device 12. For example, the connection method can be wireless, such as Bluetooth or Wi-Fi; or it can be wired, such as fiber optic, etc., without limitation. For example, electronic device 11 and notification device 12 can be connected to the Internet via a router, thereby enabling the communication connection between them.

[0043] In some implementations, the prompting device 12 is used to display the constructed hallucination assessment data. For example, the prompting device 12 may be a display device, in which case the prompting device 12 presents the hallucination assessment data to the user by displaying the hallucination assessment data on a display screen.

[0044] It should be noted that the system architecture described in the embodiments of this application is for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and does not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of system architecture, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0045] The following describes a method for constructing hallucination assessment data provided by an embodiment of this application, with reference to the accompanying drawings.

[0046] For example, the method for constructing hallucination assessment data provided in this application can be achieved through... Figure 1 The system for constructing the hallucination assessment data shown is implemented, or, through... Figure 1 The electronic device implementation in the system for constructing the hallucination assessment data shown.

[0047] See Figure 2 This is a flowchart illustrating a method for constructing hallucination assessment data provided in an embodiment of this application. Figure 2 As shown, the method for constructing hallucination assessment data provided in this application includes the following steps S201 to S204.

[0048] S201. Based on the knowledge documents of the target domain, extract at least one knowledge triple.

[0049] The knowledge triple includes a type of domain knowledge in the knowledge document, entities in the domain knowledge, and the identifier of the target domain. The domain knowledge is factual information in the knowledge document.

[0050] In some embodiments, a knowledge document can refer to text, charts, or multimedia materials containing organized and valuable information within a specific professional field or discipline. Knowledge documents typically contain validated or recognized concepts, principles, methods, facts, and data within that field.

[0051] For example, knowledge documents may include technical standard white papers for a specific field, product specifications, academic research papers, manuals or textbooks written by experts, audited project reports, and regulations or guidelines issued by authoritative organizations.

[0052] In some embodiments, knowledge documents in the target domain can be obtained by accessing a pre-built knowledge base system, a professional academic database, an authorized document management system, or a designated network resource address. When obtaining knowledge documents, filtering and locating can be performed based on keywords in the target domain, document metadata, or directory structure.

[0053] In some embodiments, domain knowledge refers to authentic information content belonging to a specific professional field or discipline. Domain knowledge reflects objectively existing facts, patterns, or consensus within that field.

[0054] For example, domain knowledge can include event-specific knowledge describing the occurrence of a specific event, such as discovery, invention, or publication; descriptive knowledge describing the attributes of an entity, such as performance, parameters, and characteristics; and relational knowledge expressing the relationships between entities, such as creation, location, and influence. For instance, the statement "Newton published *Mathematical Principles of Natural Philosophy* in 1687" fully expresses a historical event, including the subject, action, time, and object, and can serve as a direct basis for answering related questions.

[0055] In some embodiments, an entity is an object involved in domain knowledge and serves as the information carrier of knowledge statements. Entities are typically concrete or abstract things that play the role of the subject in the knowledge structure.

[0056] For example, entities can include concrete objects such as people, organizations, and geographical locations with specific names or identifiers; outputs or creations with clear names such as products, works, theories, and regulations; and abstract categories such as methods, principles, concepts, and technical terms. For instance, in the domain knowledge "Newton published Mathematical Principles of Natural Philosophy in 1687," both "Newton" and "Mathematical Principles of Natural Philosophy" are entities that carry this historical event.

[0057] In some embodiments, domain tags are data that identifies domain knowledge and the professional categories or disciplines to which its entities belong. Domain tags provide taxonomic context for knowledge, enabling it to be systematically organized and managed.

[0058] For example, domain labels may include first-level domain labels that represent broad academic disciplines, such as "physics", "history", and "computer science"; and second-level domain labels that represent more specific research directions or application scenarios, such as "classical mechanics", "history of British science", and "natural language processing".

[0059] In some embodiments, after acquiring knowledge documents in the target domain, the acquired knowledge documents can be parsed and the text extracted. The document content in non-plain text formats such as PDF and images can be converted into a unified and processable plain text format to prepare standardized input data for the subsequent knowledge triple extraction steps.

[0060] S202. Generate the first question-answer pair based on knowledge triples.

[0061] The first question-and-answer pair includes hallucinogenic questions and correct answers.

[0062] In some embodiments, a hallucinogenic question is a question specifically designed in a controlled environment to proactively induce and assess potential factual errors in a large language model. The hallucinogenic question is not designed to elicit open-ended or creative answers, but rather to construct a test case that identifies potentially factually inaccurate "hallucinations" in the model's response by comparing it to known correct responses. For example, the goal of a hallucinogenic question is to expose hallucination deficiencies in the large language model across dimensions such as entities, relations, and logic.

[0063] In some embodiments, a correct response refers to an answer to a hallucination question that is consistent with the domain knowledge in the knowledge triple. The content of a correct response reflects factual information in the knowledge document, and the correct response serves as the benchmark for determining whether a model response indicates hallucination.

[0064] In some embodiments, a knowledge triple can be used as part of the input prompt by calling a large language model generation interface that has been fine-tuned by instructions, and the large language model can be requested to automatically generate a hallucinogenic question and the corresponding correct answer based on the knowledge triple; alternatively, the entities and domain knowledge in the knowledge triple can be automatically filled into the preset question and answer sentence by executing predefined question and answer template filling rules to obtain the first question and answer pair.

[0065] In some embodiments, after generating the first question-and-answer pair, the generated first question-and-answer pair can also be checked for logical consistency and factual accuracy. By comparing the correct response with the domain knowledge in the original knowledge triple, it can be ensured that the correct response does not deviate from the original facts and that the hallucinogenic question and the correct response are logically matched.

[0066] S203. Based on the knowledge triples and the first question-answer pair, generate hallucination responses to hallucination questions, as well as hallucination type tags corresponding to the hallucination responses.

[0067] In some embodiments, hallucination responses are erroneous responses generated based on hallucinogenic questions and inconsistent with factual information in the knowledge triples. Hallucination responses typically conform to linguistic norms and the question's intent on the surface, but their content deviates factually from the original knowledge, contains logical contradictions, or introduces irrelevant information. The goal of generating hallucination responses is to provide negative examples for hallucination detection models, used to train or evaluate the model's ability to identify factual errors.

[0068] For example, hallucinatory responses can include: responses that directly contradict facts in the knowledge triple, such as altering time, place, value, or relationship; responses that are logically underivative or make additional assertions that cannot be derived from given knowledge; responses that are partially correct in form but mixed with inaccurate details; and responses that are completely detached from the knowledge context and generate content irrelevant to the question. The construction of hallucinatory responses must ensure that their errors are not due to grammatical inconsistencies or irrelevant answers, but rather stem from factual errors.

[0069] In some embodiments, hallucination type labels are data that classifies and identifies the nature of errors contained in hallucination responses. Based on a predefined classification system, hallucination type labels attribute the root causes of inconsistencies between hallucination responses and standard knowledge. Hallucination type labels provide fine-grained supervision for model training, helping to improve the accuracy of hallucination detection models in recognizing specific error patterns.

[0070] In some embodiments, based on knowledge triples and a first question-answer pair, a second question-answer generation model can be used to generate hallucinatory responses to hallucinogenic questions, as well as hallucinatory type labels corresponding to the hallucinatory responses.

[0071] S204. Construct hallucination assessment data based on at least one of the following: knowledge triples, first question-and-answer pairs, second question-and-answer pairs, and hallucination type labels.

[0072] The second question-and-answer pair includes hallucination questions and hallucination responses. Hallucination assessment data is used as sample data to build a hallucination detection model or to evaluate the severity of hallucination phenomena generated by a large language model.

[0073] In some embodiments, hallucination assessment data is a structured dataset used to train a hallucination detection model, or a dataset used to evaluate the severity of hallucination phenomena in content generated by a large language model. Hallucination assessment data consists of several interrelated data fields, including hallucinogenic question, correct response, hallucinatory response, and a hallucination type label corresponding to the hallucination response. These fields collectively describe a complete assessment instance. The design goal of hallucination assessment data is to enable hallucination detection models to identify factual errors in text responses.

[0074] In some embodiments, the hallucination detection model is a trained machine learning model specifically designed to automatically identify whether a text response contains hallucination content that contradicts or is illogical with a given source of knowledge. The input to the hallucination detection model is typically a question, a response, and optional background knowledge; the output is a judgment on whether the response contains hallucinations, or further, a classification of the hallucination type.

[0075] In some embodiments, standardized hallucination evaluation data can be obtained by automatically assembling and serializing all intermediate data products generated in the preceding steps, including knowledge triples, first question-answer pairs, second question-answer pairs, and hallucination type labels, according to a preset data structure template. The assembled data can also be saved as a file or written to a database for direct reading in subsequent model training processes.

[0076] In some embodiments, after constructing the hallucination assessment data, the assembled hallucination assessment dataset can be cleaned and deduplicated to remove samples containing invalid formats, null values, or duplicate content. Statistical analysis can be performed on the distribution of different hallucination types in the dataset, and if necessary, the class balance can be adjusted using oversampling or undersampling techniques to construct a higher quality sample dataset.

[0077] In some embodiments, step S201 can be implemented as step a1.

[0078] a1. Use triples to extract knowledge documents for the target domain from a large language model, and extract knowledge triples. The extracted knowledge triples can be divided into multiple categories, where domain knowledge, entities, and domain labels of the same category are grouped into one knowledge triple.

[0079] For example, domain knowledge and core entity extraction are performed based on a large language model extracted from triples. The input is knowledge documents in the target domain (such as authoritative knowledge bases, technical documents, books, and web pages in fields such as medicine, finance, and technology; common document formats include TXT, JSON / JSONL, and CSV). Let the document set be denoted as . ,in This represents the number of documents.

[0080] Next, large language models are extracted based on triples. It performs semantic parsing and information extraction on knowledge documents, identifying domain knowledge, entities (such as people, places, organizations, concepts, parameters, etc.) and their respective domains in the documents. It is possible to extract knowledge and entities from multiple domains from the same document. ,in, .in It is the first Domain knowledge of each sample It is the first The entities of each sample, It is the first Domain labels for each sample.

[0081] Ultimately, through the study of Domain knowledge and entity extraction are performed to form a set of structured knowledge triples. .

[0082] For example, a large language model extracted from triples can be constructed based on the following: First, the training process involves feeding the model a large number of document paragraph samples from the target domain and their corresponding labeled knowledge triple examples. This allows the model to learn and master the rules for mapping natural language descriptions to a standard "(domain knowledge, core entity, domain label)" format. For each training sample, the label is the corresponding knowledge triple, generated manually or automatically.

[0083] Secondly, the optimization objective of the model can be defined as a triplet extraction loss function, which takes into account the accuracy of the reconstruction of domain knowledge text, the correctness of the boundary recognition of core entities, and the classification accuracy of domain labels.

[0084] Then, in terms of output design, the model is trained to generate the three components in a formatted manner according to preset special characters (such as specific delimiters or tags), or directly output a structured JSON object.

[0085] Ultimately, the completed triplet extraction large language model can analyze the semantics of text and extract structured knowledge units that meet the requirements when receiving new knowledge document input.

[0086] For example, such as Figure 3 As shown, knowledge documents collected from domain knowledge bases, technical documents, books, or web pages are input into the triple extraction large language model to finally obtain knowledge triples. A knowledge triple includes domain knowledge, an entity, and the identifier of the target domain.

[0087] In some embodiments, step S202 can be implemented as step b1 or b2.

[0088] b1. Based on knowledge triples, the first question-and-answer pair can be generated by inputting the knowledge triples and the first prompt words into the first question-and-answer generation model. The first question-and-answer generation model uses the knowledge triples as contextual information and combines them with the first prompt words to generate the first question-and-answer pair.

[0089] The first cue word is used to induce the generation of hallucination questions and correct responses.

[0090] For example, knowledge triples As contextual information, it is input together with the first prompt word into the first question-answering generation model. Generate the first question-and-answer pair, namely: ,in, Referring to hallucination issues, A correct reply. The first prompt word.

[0091] In some embodiments, the first prompt word is a predefined text instruction. The first prompt word can guide and constrain the behavior of the first question-answer generation model, enabling the model to generate question-answer pairs that conform to specific format and content requirements based on given knowledge triples.

[0092] For example, the first prompt can clearly define the rules for generating the first question and answer pair, ensuring the difficulty of the hallucinogenic question and the accuracy of the correct response. The first prompt can include: the generated question must be of a certain difficulty, avoiding simple factual restatements (such as avoiding closed-ended questions like "Did Newton publish *Mathematical Principles of Natural Philosophy*?" and prioritizing multi-dimensional questions like "When did Newton publish *Mathematical Principles of Natural Philosophy* and what are the core contributions of this work?"); the question must be strongly related to the knowledge content in the knowledge triple and must not generate irrelevant questions detached from background knowledge; the correct response must completely and accurately address the hallucinogenic question and be strictly based on the domain knowledge and entities in the knowledge triple, without introducing external information; the correct response must conform to factual logic, without ambiguity or contradiction (such as time, numerical values, and relational descriptions must be completely consistent with the knowledge content).

[0093] In some embodiments, the first question-answer generation model refers to a large language model that has been fine-tuned with instructions and is specifically designed to generate hallucinogenic questions and correct responses. The role of the first question-answer generation model is to transform structured knowledge triples into first question-answer pairs in natural language form.

[0094] For example, the first question-answer generation model can be constructed based on the following: First, the training data for the first question-answering generation model consists of input-output paired samples. The input samples are composed of two parts: a designed prompt word template and specific knowledge triples, filled into the template in a structured or natural language description. The tags are corresponding, manually written or verified question-answer pairs, i.e., a natural language question and its answer targeting domain knowledge.

[0095] Secondly, a pre-trained language model can be chosen as the base, such as a model based on a Decoder-only architecture or an Encoder-Decoder architecture. The model's input format is designed to simultaneously accept prompt words and knowledge triples, and output question-answer pairs.

[0096] Finally, supervised fine-tuning is used to train the pedestal model. During training, the model learns the mapping from "cue word + knowledge triple" to "complete question-answer pair". The loss function is typically a standard next-word prediction loss (such as cross-entropy loss), and the model optimizes by minimizing the difference between the generated question-answer pair text and the labeled text in the training data.

[0097] For example, such as Figure 4 As shown, the knowledge triples and the first prompt word are input into the first question-and-answer generation model, which will eventually output the first question-and-answer pair. The first question-and-answer pair includes the hallucinogenic question and the correct answer to the hallucinogenic question.

[0098] b2. Based on the knowledge triples, generate the first question-and-answer pair. Alternatively, by executing predefined question-and-answer template filling rules, the entities and domain knowledge in the knowledge triples can be automatically filled into the preset question and response sentences to obtain the first question-and-answer pair.

[0099] For example, firstly, based on the characteristics of the knowledge triples, a matching predefined question-and-answer template is selected from the template library. Then, according to rules, entity and domain knowledge is extracted and parsed from the knowledge triples and filled into placeholders in the question sentence to form a complete hallucinogenic question; simultaneously, the parsed entity or domain knowledge text is filled into placeholders in the response sentence to form a complete correct response. Finally, the generated question and response are combined and output as the first question-and-answer pair.

[0100] In some embodiments, the question phrase is a fixed sentence structure within a predefined question-and-answer template specifically designed to construct the hallucinogenic question section. The question phrase has built-in specific grammar and interrogative words, and includes positional markers for embedding specific content of knowledge triples.

[0101] For example, question formats may include: attribute-based questions with the structure "What is [a certain attribute] of [entity]?"; relationship-based questions with the structure "What is the relationship between [entity] and [entity]?"; or open-ended questions with the structure "Please describe the role of [entity] in [an event]". [] represents placeholders.

[0102] In some embodiments, the response phrase is a fixed sentence structure within a predefined question-and-answer template specifically designed to construct the correct response. The response phrase provides a standard way of stating facts and includes position markers for filling in specific factual content derived from knowledge triples that corresponds to the question.

[0103] For example, response formats may include: attribute statement formats with the structure "[entity]'s [a certain attribute] is [attribute content]"; relation statement formats with the structure "[entity] and [entity] have a [relationship]", etc. Here, [] represents placeholders.

[0104] In some embodiments, the fill rules are a logically defined set of instructions that define how to map and replace the different components of a knowledge triple into corresponding placeholders in a predefined question-answer template. The fill rules specify the transformation logic from structured knowledge to natural language sentences, ensuring that the final synthesized question-answer pairs are semantically consistent with the original knowledge.

[0105] For example, the filling rules may include: a rule that "replaces the [entity] placeholder in the question and response statements with the entity field value in the knowledge triple"; a rule that "replaces the [attribute content] placeholder in the response statement with the domain knowledge field value in the knowledge triple, either as a whole or after partial parsing"; and a conditional logic rule that selects different question-and-answer template variants based on the knowledge type, etc.

[0106] In some embodiments, step S203 can be implemented as step c1.

[0107] c1. Input the knowledge triple, the first question-and-answer pair, the preset hallucination type, and the second prompt word into the second question-and-answer generation model. The second question-and-answer generation model uses the knowledge triple as contextual information and combines it with the first question-and-answer pair, the preset hallucination type, and the second prompt word to generate a hallucination response to the hallucination question and a hallucination type label corresponding to the hallucination response. The second prompt word is used to induce the generation of a hallucination response to the hallucination question, and the hallucination response satisfies the preset hallucination type.

[0108] For example, knowledge triples As contextual information, related to hallucination issues Correct reply Hallucination response prompts and preset hallucination types Common Input Second Question Answering Generation Model This generates a corresponding hallucinatory response. And corresponding hallucination response tags ,Right now:

[0109] In some embodiments, a preset hallucination type is a predefined and enumerated set of standard categories used to classify and describe the causes of factual errors. The preset hallucination type provides explicit error direction constraints and classification criteria for the generation of hallucination responses, ensuring that the generated error samples can systematically cover hallucination phenomena of different natures.

[0110] In some embodiments, the preset illusion type includes at least one of the following: entity error illusion, relation error illusion, attribute error illusion, fact fabrication illusion, instruction inconsistency illusion, context inconsistency illusion, and logic inconsistency illusion.

[0111] For example, entity error illusion is an error in the description of the identity, name, existence, etc. of real-world entities (such as people, places, organizations, concepts, etc.), including entity confusion, entity non-existence, etc.

[0112] For example, the illusion of relational error is a misrepresentation of the relationships between entities (such as causal, subordinate, spatiotemporal, etc.) that contradicts the true logical or factual connections between entities.

[0113] For example, the attribute error illusion is an incorrect description of an entity's characteristics and attributes (such as time, quantity, properties, state, etc.) that does not match the entity's true attributes.

[0114] For example, factual fiction illusions are the generation of facts, events, or concepts that have no basis in reality.

[0115] For example, the illusion of inconsistency in instructions is that the generated content does not follow the user's explicit instructions, deviates from the task objectives or format specifications.

[0116] For example, the context inconsistency illusion occurs when the generated content contradicts the input knowledge triplet, and the content is inconsistent with itself.

[0117] For example, the illusion of logical inconsistency is that there are logical contradictions within the generated content, or that the reasoning process violates basic logical rules (such as reversed cause and effect, self-contradiction, etc.).

[0118] In some embodiments, the second prompt is a predefined text instruction. The second prompt can guide and constrain the behavior of the second question-answering generation model, enabling the model to generate seemingly reasonable hallucinatory responses that conform to a specified error pattern, based on a full understanding of the knowledge triples, hallucinogenic questions, and correct responses.

[0119] For example, the second prompt word could be a clear rule for generating hallucination responses to ensure the validity and accuracy of the labeling, including: the generated hallucination response must be strongly related to the hallucinogenic question and must not generate content unrelated to the question; the generated hallucination response must be inconsistent with the domain knowledge in the knowledge triplet, i.e., it must violate the factual basis; the generated hallucination response must be inconsistent with the correct response, and the difference must correspond to a clear type of hallucination; the generated hallucination response only contains the response itself and must not be accompanied by explanatory text such as "this is the wrong answer" or "hallucination exists here".

[0120] In some embodiments, the second question-answering generation model refers to a large language model that has been fine-tuned with instructions specifically for generating hallucination responses. The role of the second question-answering generation model is to synthesize a hallucination response that meets the preset hallucination type requirements based on the prompting of the second prompt word and in combination with all the contextual information of the input, and to correctly classify it.

[0121] For example, the second question-answering generation model can be constructed based on the following: First, the input samples for the model's training data include knowledge triples, a first question-answer pair (containing a hallucinogenic question and a correct response), a preset hallucination type, and a second cue word. The labels are the corresponding, manually written or verified hallucination responses and the hallucination type labels that match those responses.

[0122] Then, the structure and training method of the second question-answer generation model can be set up with reference to the first question-answer generation model.

[0123] For example, such as Figure 5 As shown, the knowledge triple, the first question-and-answer pair, the second prompt word, and the preset hallucination type are input into the second question-and-answer generation model, and the final output is the hallucination response and the hallucination type label.

[0124] In some embodiments, such as Figure 6 As shown in the illustration, the generation module provided in this application can generate the required hallucination assessment data. First, through the domain knowledge and entity extraction module, knowledge triples can be generated from the knowledge document. Next, through the hallucination question and correct response generation module, a first question-and-answer pair can be generated based on the knowledge triples. Finally, through the hallucination question and hallucination response generation module, hallucination responses and hallucination type tags can be generated based on the knowledge triples and the first question-and-answer pair, which together constitute the hallucination assessment data.

[0125] In some embodiments, the hallucination assessment data generated by the above method may contain low-quality samples such as "invalid high-hallucination problem," "inaccurate correct response," "substandard hallucination response," and "incorrect labeling." Therefore, after performing step S204, as follows... Figure 7 As shown, the validity of the hallucination assessment data can also be verified by performing the following steps S701~S702, and invalid items can be corrected if there are invalid items in the hallucination assessment data.

[0126] S701. Validate the hallucination assessment data based on the validation model set.

[0127] The validation model set includes multiple validation models with different structures.

[0128] In some embodiments, the validation model set is a collection of machine learning models with different architectures, training objectives, or knowledge backgrounds. Each member model in the validation model set has the ability to evaluate the factual consistency or logical plausibility of the text. The role of the validation model set is to provide multi-faceted and complementary evaluation perspectives, comprehensively assessing the validity of hallucination evaluation data through independent and potentially divergent analyses of the same hallucination evaluation data.

[0129] For example, the set of verification models may include a deterministic verification model based on rule or knowledge graph matching to judge the factual accuracy of the response based on structured knowledge; a pre-trained language model to evaluate the authenticity of the statement through semantic understanding; and a large language model to directly judge whether the response is a hallucination through instruction-induced judgment.

[0130] In some embodiments, the verification model set includes at least one of the following: a hallucination problem verification model set, a correct response verification model set, a hallucination response verification model set, and a hallucination label verification model set.

[0131] In some embodiments, various verification model sets can be verified through a voting mechanism of the models.

[0132] For example, each validation model independently outputs a validity judgment result. As shown below.

[0133]

[0134] in, Indicates the first The content to be verified for each sample. This indicates the first [number] model in the current category of the validation model set. A validation model, Indicates the first One sample, , This indicates that the verification is invalid. The time indicates that the verification is valid. A "majority voting" strategy is adopted, which integrates the verification results of multiple large models for the current type of verification model and votes on them. If more than half of the models are deemed valid, the final verification result is [validity]. 1 indicates the verification is valid; otherwise... The value '0' indicates that the verification is invalid. The verification result can be shown below.

[0135]

[0136] in, Indicates the first Validation results of one sample This indicates the number of models in the current type of validation model set.

[0137] In some embodiments, different sets of verification models can implement different verification functions for different content in the hallucination assessment data.

[0138] For example, a hallucinogenic question verification model is used to verify whether a hallucinogenic question meets a preset hallucinogenic question generation rule.

[0139] For example, the correct response verification model is used to verify whether a correct response meets the preset correct response generation rules.

[0140] For example, the hallucination response verification model is used to verify whether the hallucination response meets the preset hallucination response generation rules.

[0141] For example, the hallucination label verification model is used to verify whether the hallucination type label is consistent with the characteristics of the hallucination response.

[0142] S702. If there are invalid items in the hallucination assessment data, the invalid items shall be corrected.

[0143] In some embodiments, invalid items can be corrected based on a correction model; wherein the correction model includes at least one of the following: a hallucination problem correction model, a correct response correction model, a hallucination response correction model, and a hallucination label correction model.

[0144] In some embodiments, the correction model is a type of machine learning model specifically designed to correct specific types of errors in hallucination assessment data. The correction model receives invalid data items, or error diagnosis information corresponding to erroneous items as input, and outputs corrected, compliant data content by understanding the original generation intent and quality rules.

[0145] In some embodiments, when the invalid item is a hallucinogenic problem, the invalid item is corrected based on a knowledge triple.

[0146] For example, if the hallucination problem verification model set does not verify that the hallucination problem meets the preset hallucination problem generation rules, it is necessary to perform hallucination problem correction by inputting the knowledge triple and the hallucination problem to be corrected into the hallucination problem correction model. Alternatively, you can input the reasons for invalid verification provided by the hallucination problem verification model to generate a corrected hallucination problem. The hallucinogenic issue can be corrected as follows:

[0147] in, Represents a knowledge triple. This indicates a hallucinogenic issue that needs to be addressed.

[0148] For example, a hallucination problem correction model can be constructed based on the following: First, the training data consists of a large number of targeted correction samples. Each sample may include a knowledge triple and a hallucinogenic question to be corrected, or it may include a description of the specific rule violation that makes the hallucinogenic question invalid, provided by the validation model (e.g., "the question is not a question, the difficulty is insufficient"), as well as a corrected hallucinogenic question that satisfies all preset rules. These elements are concatenated according to a template to form the input sequence of the model, and the corresponding label is the corrected hallucinogenic question text.

[0149] Then, the constructed training data is input into a base model. The base model can be a large language model that excels in natural language understanding and instruction following, possessing powerful text generation and rewriting capabilities. During training, the model learns the mapping relationship from "the context containing knowledge triples and the hallucinogenic problem to be corrected" to "the corrected hallucinogenic problem." The loss function calculates the difference between each word generated by the model and the labeled word of the corrected hallucinogenic problem. Through supervised fine-tuning, the model not only rewrites sentences but also internalizes the understanding of preset generation rules (such as difficulty and relevance), thus enabling targeted semantic reconstruction and optimization of the problem to be corrected based on specific reasons for ineffectiveness.

[0150] For example, such as Figure 8 As shown, the knowledge triples and the hallucinogenic question are input into the hallucinogenic question verification model for verification. If the verification fails, the knowledge triples and the hallucinogenic question are input into the hallucinogenic question correction model for correction, resulting in the knowledge triples and the corrected hallucinogenic question.

[0151] In some embodiments, invalid items are corrected based on knowledge triples and hallucination issues when the invalid item is a correct response.

[0152] For example, if the correct response verification model set does not meet the preset correct response generation rules, correct response correction needs to be performed, and the knowledge triplet needs to be corrected. Hallucination issues Correct reply to be corrected Correct input response correction model Alternatively, you can input the reason for invalid verification given by the correct response verification model to generate a corrected response. A correct reply can be corrected as follows:

[0153] For example, a correct response correction model can be constructed based on the following: First, the training data consists of a large number of targeted correction samples. Each sample can include knowledge triples, hallucinogenic questions, and correct responses to be corrected. It can also include descriptions of specific rule violations that invalidate correct responses, provided by the validation model (e.g., "factual error: the time '1787' in the response does not match '1687' in the knowledge") or "incomplete answer: the core contribution of the work is not stated"), as well as corrected correct responses that satisfy all preset rules. These elements are concatenated according to a template to form the input sequence of the model, and the corresponding labels are the corrected correct response texts.

[0154] Then, the constructed training data is input into a base model. The base model can be a large language model that excels in natural language understanding and instruction following, possessing powerful text generation and rewriting capabilities. During training, the model learns the mapping relationship from "context containing knowledge triples, hallucinogenic questions, and the correct response to be corrected" to "corrected correct response." The loss function calculates the difference between each lexical unit generated by the model and the labeled corrected correct response lexical unit. Through supervised fine-tuning, targeted semantic reconstruction and optimization of the correct response to be corrected can be performed based on specific reasons for ineffectiveness.

[0155] For example, such as Figure 9 As shown, the knowledge triple, the hallucinogenic question, and the correct response are input into the correct response verification model for verification. If the verification fails, the knowledge triple, the hallucinogenic question, and the correct response are input into the correct response correction model for correction, resulting in the knowledge triple, the hallucinogenic question, and the corrected correct response.

[0156] In some embodiments, when the invalid item is a hallucinatory response, the invalid item is corrected based on the knowledge triple, the hallucinogenic problem, and the correct response.

[0157] For example, if the hallucination response verification model set fails to meet the preset hallucination response generation rules, hallucination response correction needs to be performed, and the knowledge triplet needs to be corrected. Hallucination issues Correct reply And the hallucination response to be corrected Input hallucination response correction model Alternatively, you can input the reason for invalid verification given by the hallucination response verification model to generate a corrected hallucination response. The hallucination recovery correction can be done as follows:

[0158] For example, the hallucination response correction model can be constructed based on the following: First, the training data consists of a large number of targeted correction samples. Each sample can include knowledge triples, hallucinogenic questions, correct responses, and hallucinatory responses to be corrected. It can also include descriptions of specific rule violations that invalidate the hallucinatory responses, provided by the validation model, as well as corrected hallucinatory responses that satisfy all preset rules. These elements are concatenated according to a template to form the input sequence of the model, and the corresponding labels are the corrected hallucinatory response texts.

[0159] Then, the constructed training data is input into a base model. The base model can be a large language model that excels in natural language understanding and instruction following, possessing powerful text generation and rewriting capabilities. During training, the model learns the mapping relationship from "context containing knowledge triples, hallucinogenic questions, correct responses, and the hallucinatory response to be corrected" to "corrected hallucinatory responses." The loss function calculates the difference between each lexical unit generated by the model and the labeled lexical unit of the corrected hallucinatory response. Through supervised fine-tuning, targeted semantic reconstruction and optimization of the hallucinatory responses to be corrected can be performed based on specific reasons for ineffectiveness.

[0160] For example, such as Figure 10 As shown, the knowledge triple, hallucinogenic question, correct response, and hallucinatory response are input into the hallucinatory response verification model for verification. If the verification fails, the knowledge triple, hallucinogenic question, correct response, and hallucinatory response are input into the hallucinatory response correction model for correction, resulting in the knowledge triple, hallucinogenic question, correct response, and corrected hallucinatory response.

[0161] In some embodiments, when the invalid item is a hallucination type label, the invalid item is corrected based on the knowledge triple, hallucination question, correct response, and hallucination response.

[0162] For example, if the hallucination label verification model set verifies that the hallucination type label is an incorrect hallucination type label, label correction needs to be performed to correct the hallucination problem. Correct reply Hallucination recovery With tags to be corrected Input illusion label correction model Alternatively, you can input the reason for invalid verification given by the hallucination label verification model to generate a corrected hallucination type label. The hallucination type label can be corrected as follows:

[0163] For example, the hallucination response correction model can be constructed based on the following: First, the training data consists of a large number of targeted correction samples. Each sample may include a hallucination question, a correct response, a hallucination response, and a hallucination type label to be corrected. It may also include a description of the specific inconsistencies given by the validation model that cause the hallucination type label to be invalid, as well as a corrected hallucination type label that matches the hallucination response characteristics. These elements are concatenated according to a template to form the input sequence of the model, and the corresponding label is the corrected hallucination type label.

[0164] Then, the constructed training data is input into a base model. The base model can be a large language model that excels in natural language understanding and instruction following, possessing powerful text generation and rewriting capabilities. During training, the model learns the mapping relationship from "context containing hallucinogenic questions, correct responses, hallucinatory responses, and hallucinatory type labels to be corrected" to "corrected hallucinatory type labels." The loss function calculates the difference between each word generated by the model and the labeled corrected hallucinatory type label word. Through supervised fine-tuning, targeted corrections to the hallucinatory type labels can be made.

[0165] For example, such as Figure 11 As shown, the hallucination question, correct response, hallucination response, and hallucination type label are input into the hallucination label verification model for verification. If the verification fails, the hallucination question, correct response, hallucination response, and hallucination type label are input into the hallucination label correction model for correction, resulting in the hallucination question, correct response, hallucination response, and corrected hallucination type label.

[0166] In some embodiments, the validity of the corrected invalid items can be verified by inputting the correction results into a verification model set for validity verification.

[0167] In some embodiments, if the correction result is verified to be valid, the correction ends; if the correction result is verified to be invalid, the correction is iterated.

[0168] The iterative correction includes iteratively correcting the result of the previous correction.

[0169] In some embodiments, iterative correction refers to a repetitive or incremental correction loop process initiated when a single correction operation fails to make an invalid item pass validity verification. The output of the previous correction operation is used as the input for the current iteration, and the same correction strategy is applied again until a valid result that passes verification is produced, or a preset termination condition is met.

[0170] For example, the preset termination condition can be set to a maximum number of corrections.

[0171] In some embodiments, if the verification fails after reaching the maximum number of corrections, the entire sample, including hallucination questions, correct responses, hallucination responses, and hallucination type labels, is discarded.

[0172] In some embodiments, step S701, which verifies the validity of the hallucination assessment data based on the verification model set, can be implemented as the following steps d1 to d4.

[0173] d1. Use at least a preset number of hallucinogenic question verification models to verify whether the hallucinogenic questions meet the preset hallucinogenic question generation rules.

[0174] In some embodiments, the preset hallucinogenic question generation rule is a predefined text instruction that can guide and constrain the generation of hallucinogenic questions that conform to specific format and content requirements based on a given knowledge triple.

[0175] For example, the pre-defined rules for generating hallucinogenic questions may include requiring the generated questions to be of a certain difficulty, avoiding simple factual restates (such as avoiding closed-ended questions like "Did Newton publish 'Mathematical Principles of Natural Philosophy'?" and prioritizing multi-dimensional questions like "When did Newton publish 'Mathematical Principles of Natural Philosophy' and what are the core contributions of the work?"); the questions must be strongly related to the knowledge content in the knowledge triplet and must not generate irrelevant questions that are divorced from background knowledge, etc.

[0176] In some embodiments, the hallucination question verification model receives a knowledge triple and a hallucination question to be verified as input, and evaluates whether the question meets the preset generation rules through an internal multi-layer attention mechanism and semantic understanding module.

[0177] For example, the hallucination question verification model first parses the knowledge triples, extracting information about core entities and domain knowledge to form an internal factual representation. Next, the verification model analyzes the syntactic structure, interrogative word usage, and semantic focus of the hallucination question to determine if it is a closed-ended question of simple fact restatement. The verification model assesses the semantic correlation between the question's intent and the coverage of the knowledge content, ensuring the question remains within the context of the knowledge. Finally, the verification model outputs a structured verification result, which can include not only a binary judgment of pass or fail but also a quantitative score of the question's difficulty level and relevance, and may generate specific modification suggestions, such as pointing out overly simplistic parts of the question or deviations from the intended knowledge.

[0178] For example, a hallucination verification model can be constructed based on the following: First, the training data consists of a large number of labeled samples. Each sample includes a knowledge triple, a hallucinogenic question, and a rule for generating the hallucinogenic question, which are concatenated according to a specific template. The label corresponding to the sample not only includes the overall judgment of "pass / fail", but can also be refined to specify which rule was violated (e.g., "insufficient difficulty: belongs to a yes / no question"), and can include brief modification suggestions or confidence scores.

[0179] Then, the constructed training data is input into the base model. During training, the model learns the mapping relationship from "input with rule instructions and data" to "structured output," and the loss function calculates the difference between the word units generated by the model and the labeled standard output word units. The base model can be a large language model that excels in natural language understanding and instruction following.

[0180] d2. Use at least a preset number of correct responses to verify whether the correct responses meet the preset correct response generation rules.

[0181] In some embodiments, the preset correct response generation rule is a predefined text instruction that can guide and constrain the generation of correct responses that meet specific format and content requirements based on a given knowledge triple and a generated hallucinogenic question.

[0182] For example, the preset rules for generating correct responses may include that correct responses must fully and accurately address the question, and be generated strictly based on domain knowledge and entities in the knowledge triple, without introducing external information; correct responses must conform to factual logic, and be unambiguous and consistent (e.g., time, numerical values, and relationship descriptions must be completely consistent with the knowledge content).

[0183] In some embodiments, the correct response verification model receives a knowledge triple, a hallucinogenic question, and a correct response to be verified as input, and evaluates whether the question meets the preset generation rules through an internal multi-layer attention mechanism and semantic understanding module.

[0184] For example, the correct response verification model first parses the knowledge triples, extracting factual information such as entities, attributes, and relationships to form an internal factual baseline. Simultaneously, the model analyzes the hallucinogenic question, clarifying its inquiring intent, the key elements of the required answer, and its logical scope. Next, the verification model analyzes the correct response to be verified, understanding its semantic structure and identifying the stated factual points, logical relationships, and responses to the question. The correct response verification model then performs cross-comparison: on the one hand, it matches each factual assertion in the response with the factual baselines in the knowledge triples, performing item-by-item matching and implication relationship reasoning to ensure the response has not been altered, omitted, or added any additional factual details; on the other hand, it aligns the content of the response with all the key points of the hallucinogenic question, checking whether the response comprehensively and unambiguously addresses the question, and whether there are any irrelevant or evasive answers.

[0185] For example, a correct response validation model can be constructed based on the following: First, the training data consists of a large number of labeled samples. Each sample includes a knowledge triple, a hallucinogenic question, a correct response to be verified, and a pre-defined text for generating correct responses, all concatenated according to a specific instruction template. The labels corresponding to the samples not only include an overall "pass / fail" judgment, but can also be refined to specify which rule was violated (e.g., "factual error: the time in the response does not match the knowledge"), and can include correction suggestions.

[0186] Then, the constructed training data is input into the base model. The base model can be a large language model that excels in natural language understanding and instruction following. During training, the model learns the mapping relationship from "input with rule instructions, knowledge, questions, and responses" to "structured validation output." The loss function calculates the difference between each word generated by the model and the labeled standard output word. Through supervised fine-tuning, the model gradually internalizes the semantics of the preset rules and learns how to evaluate whether a correct response meets the preset correct response generation rules.

[0187] d3. Use at least a preset number of hallucination response verification models to verify whether the hallucination responses meet the preset hallucination response generation rules.

[0188] In some embodiments, the preset hallucination response generation rule can be a predefined text instruction that can guide and constrain the generation of seemingly reasonable hallucination responses that conform to a specified error pattern based on knowledge triples, hallucinogenic questions, and correct responses.

[0189] For example, the preset hallucination response generation rules may include: the generated hallucination response must be strongly related to the hallucinogenic question and must not generate content unrelated to the question; the generated hallucination response must be inconsistent with the domain knowledge in the knowledge triplet, that is, it must violate the factual basis; the generated hallucination response must be inconsistent with the correct response, and the difference must correspond to a clear hallucination type; the generated hallucination response only contains the response itself and must not be accompanied by explanatory text such as "this is the wrong answer" or "there is a hallucination here".

[0190] In some embodiments, each hallucination response verification model independently receives complete contextual information, including knowledge triples, hallucination questions, correct responses, preset hallucination types, and the hallucination response to be verified. Each model outputs a structured verification result based on its internal logic, which includes at least a judgment on whether each generation rule is met, and an overall conclusion.

[0191] For example, a hallucination response verification model can be constructed based on the following: First, the training data consists of a large number of finely labeled samples. Each sample includes a knowledge triple, a hallucinogenic question, a correct response, a preset hallucination type, a hallucination response to be verified, and the complete text of the preset hallucination response generation rule. All of this information is concatenated according to a designed instruction template, which distinguishes each element. The label corresponding to the sample is a structured output that not only includes a judgment of "overall pass or fail," but also a judgment of compliance with each specific rule (such as question relevance, factual violation, type alignment, and formatting), and can further indicate the specific manifestations of rule violations and suggested directions for correction.

[0192] Then, the constructed training data is input into the base model. The base model can be a large language model that excels in natural language understanding and complex instruction following. During training, the model learns the mapping relationship from "input containing complete context and rule instructions" to "structured validation reports". The loss function calculates the difference between each word generated by the model and the labeled standard output word containing multi-dimensional judgments. Through supervised fine-tuning, the model gradually internalizes the semantics of the preset rules and learns how to evaluate whether the hallucination response meets the preset hallucination response generation rules.

[0193] d4. Use at least a preset number of hallucination label verification models to verify whether the hallucination type label and the hallucination response features are consistent.

[0194] In some embodiments, each hallucination label verification model independently receives complete contextual information, including the hallucination question, correct response, hallucination response, and the hallucination type label to be verified. Each model outputs a structured verification result based on its internal logic, which includes at least a consistency judgment result (such as "consistent" or "inconsistent") and possible reasons (such as "The response error is attribute tampering, which does not match the 'entity error' label, and it is recommended to change it to 'attribute error'").

[0195] For example, an illusion label verification model can be constructed based on the following: First, the training data consists of a large number of samples. Each sample includes: a hallucination question, a correct response, the hallucination response text, and a hallucination type label to be verified. The label corresponding to the sample is a structured output containing a "consistency judgment" (consistent / inconsistent) and a "reason for inconsistency analysis" (e.g., where the mismatch occurs and what type it should be).

[0196] Then, the constructed training data is input into a base model, which can be a large language model with powerful text understanding and reasoning capabilities. During training, the model learns a complex mapping from inputs of "hallucination question, correct response, hallucination response, label" to "consistency analysis." The loss function considers both the accuracy of consistency classification and the quality of causal analysis generation. Through supervised fine-tuning, the model internalizes the discrimination boundaries of various hallucination types and learns to associate text features with abstract type definitions, ultimately verifying whether the hallucination type label and the hallucination response features are consistent.

[0197] In some embodiments, such as Figure 12 As shown. The verification and correction module provided in this application embodiment can correct the generated hallucination assessment data. First, the hallucination problem verification and correction module verifies the hallucination problem, and corrects the hallucination problem if the verification is invalid; then, the correct response verification and correction module verifies the correct response, and corrects the correct response if the verification is invalid; next, the hallucination response verification and correction module verifies the hallucination response, and corrects the hallucination response if the verification is invalid; finally, the hallucination type label verification and correction module verifies the hallucination type label, and corrects the hallucination type label if the verification is invalid.

[0198] As can be seen, the above mainly describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, the embodiments of this application provide corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the modules and algorithm steps of the various examples described in the embodiments disclosed herein, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention.

[0199] This application embodiment can divide the hallucination assessment data construction device into functional modules according to the above method example. For example, each function can be divided into a separate functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. Optionally, the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0200] In some embodiments, this application also provides an apparatus for constructing hallucination assessment data. This apparatus may include one or more functional modules for implementing the hallucination assessment data construction method of the above method embodiments.

[0201] For example, Figure 13 This is a schematic diagram illustrating the composition of an apparatus 800 for constructing hallucination assessment data, provided in an embodiment of this application. Figure 13 As shown, the apparatus for constructing the hallucination assessment data includes an extraction module 801 and a processing module 802.

[0202] Extraction module 801 is used to extract at least one knowledge triple from knowledge documents in the target domain. Processing module 802 is used to generate a first question-answer pair based on the knowledge triple; generate a second question-answer pair based on the knowledge triple and the first question-answer pair; and construct sample data for the hallucination annotation model based on the second question-answer pair and the tags corresponding to the hallucination responses.

[0203] In some embodiments, the processing module 802 is further configured to input the knowledge triple and the first prompt word into the first question-answer generation model, and use the knowledge triple as context information and combine it with the first prompt word to generate a first question-answer pair; wherein, the first prompt word is used to induce the generation of illusion questions and correct responses.

[0204] In other embodiments, the processing module 802 is further configured to input a type of domain knowledge and entity from the knowledge triple into a preset question and response sentence to obtain a first question-answer pair.

[0205] In some other embodiments, the processing module 802 is further configured to input the knowledge triple, the first question-and-answer pair, the preset hallucination type, and the second prompt word into the second question-and-answer generation model, and use the knowledge triple as context information through the second question-and-answer generation model, and combine the first question-and-answer pair, the preset hallucination type, and the second prompt word to generate a hallucination response to the hallucination question, as well as a hallucination type label corresponding to the hallucination response; wherein, the second prompt word is used to induce the generation of a hallucination response to the hallucination question, and the hallucination response satisfies the preset hallucination type.

[0206] In some other embodiments, the preset illusion type includes at least one of the following: entity error illusion, relation error illusion, attribute error illusion, fact fabrication illusion, instruction inconsistency illusion, context inconsistency illusion, and logic inconsistency illusion.

[0207] In some other embodiments, the processing module 802 is also used to verify the validity of the hallucination assessment data; and to correct invalid items if invalid items exist in the hallucination assessment data.

[0208] In the case of implementing the functions of the integrated modules described above in hardware, this embodiment of the invention provides a possible structural schematic diagram of the electronic device involved in the above embodiments. For example... Figure 14 As shown, the electronic device 900 includes: a processor 902, a communication interface 903, and a bus 904. Optionally, the electronic device 900 may also include a memory 901.

[0209] Processor 902 may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 902 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 902 may also be a combination that implements computing functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0210] The communication interface 903 is used to connect to other devices via a communication network. This communication network can be Ethernet, wireless access network, wireless local area network (WLAN), etc.

[0211] The memory 901 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0212] In one possible implementation, the memory 901 can exist independently of the processor 902. The memory 901 can be connected to the processor 902 via a bus 904 and is used to store instructions or program code. When the processor 902 calls and executes the instructions or program code stored in the memory 901, it can implement the method for constructing hallucination evaluation data provided in this embodiment of the invention.

[0213] In another possible implementation, the memory 901 can also be integrated with the processor 902.

[0214] The 904 bus can be an extended industry standard architecture (EISA) bus, etc. The 904 bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 14 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0215] Through the above description of the implementation methods, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the service calling device can be divided into different functional modules to complete all or part of the functions described above.

[0216] This application also provides a computer-readable storage medium. All or part of the processes in the above method embodiments can be executed by computer instructions instructing related hardware. The program can be stored in the aforementioned computer-readable storage medium, and when executed, it can include the processes of the above method embodiments. The computer-readable storage medium can be any of the foregoing embodiments or memory. The aforementioned computer-readable storage medium can also be an external storage device of the aforementioned service invocation device, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the aforementioned service invocation device. Further, the aforementioned computer-readable storage medium can include both internal storage units of the aforementioned service invocation device and external storage devices. The aforementioned computer-readable storage medium is used to store the aforementioned computer program and other programs and data required by the aforementioned service invocation device. The aforementioned computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0217] This application also provides a computer program product comprising a computer program that, when run on a computer, causes the computer to execute any of the methods for constructing illusion assessment data provided in the above embodiments.

[0218] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for constructing hallucination assessment data, characterized in that, The method includes: Based on a knowledge document of the target domain, at least one knowledge triple is extracted. The knowledge triple includes a type of domain knowledge in the knowledge document, an entity in the domain knowledge, and an identifier of the target domain. The domain knowledge is factual information in the knowledge document. Based on the knowledge triples, a first question-and-answer pair is generated, which includes a hallucinogenic question and a correct answer. Based on the knowledge triplet and the first question-answer pair, a hallucination response to the hallucination question and a hallucination type tag corresponding to the hallucination response are generated. Hallucination assessment data is constructed based on at least one of the knowledge triples, the first question-and-answer pair, the second question-and-answer pair, and the hallucination type label. The second question-and-answer pair includes the hallucinogenic question and the hallucination response. The hallucination assessment data is used to construct sample data for a hallucination detection model or to evaluate the severity of hallucination phenomena generated by a large language model.

2. The method according to claim 1, characterized in that, The generation of the first question-answer pair based on the knowledge triples includes: The knowledge triple and the first prompt word are input into the first question-and-answer generation model. The first question-and-answer generation model uses the knowledge triple as context information and combines it with the first prompt word to generate the first question-and-answer pair. The first prompt word is used to induce the generation of hallucination questions and correct answers.

3. The method according to claim 1, characterized in that, The generation of the first question-answer pair based on the knowledge triples includes: The domain knowledge and the entity in the knowledge triple are input into a preset question and response format to obtain the first question-answer pair.

4. The method according to claim 1, characterized in that, The step of generating a hallucinatory response to the hallucination question based on the knowledge triple and the first question-answer pair, and a hallucination type tag corresponding to the hallucination response, includes: The knowledge triple, the first question-and-answer pair, the preset hallucination type, and the second prompt word are input into the second question-and-answer generation model. The second question-and-answer generation model uses the knowledge triple as context information and combines it with the first question-and-answer pair, the preset hallucination type, and the second prompt word to generate a hallucination response to the hallucination question and a hallucination type label corresponding to the hallucination response. The second prompt word is used to induce the generation of a hallucination response to the hallucination question, and the hallucination response satisfies the preset hallucination type.

5. The method according to claim 4, characterized in that, The preset hallucination type includes at least one of the following: Illusions of entity error, illusion of relation error, illusion of attribute error, illusion of factual fabrication, illusion of inconsistent instructions, illusion of inconsistent context, and illusion of inconsistent logic.

6. The method according to claim 1, characterized in that, The method further includes: The validity of the hallucination assessment data was verified. If invalid entries exist in the hallucination assessment data, the invalid entries are corrected.

7. A device for constructing hallucination assessment data, characterized in that, include: Extraction module, processing module; The extraction module is used to extract at least one knowledge triple based on knowledge documents in the target domain; The processing module is used to generate a first question-and-answer pair based on knowledge triples; generate a second question-and-answer pair based on the knowledge triples and the first question-and-answer pair; and construct sample data for a hallucination annotation model based on the second question-and-answer pair and the tags corresponding to the hallucination responses.

8. An electronic device, characterized in that, The device includes a processor and a memory, the processor being coupled to the memory; the memory is used to store computer instructions, which are loaded and executed by the processor to enable the computer device to implement the method for constructing hallucination assessment data as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer-executable instructions that, when executed on a computer, cause the computer to perform a method for constructing hallucination assessment data as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when run on an electronic device, causes the electronic device to perform a method for constructing hallucination assessment data as described in any one of claims 1 to 6.

Images