Knowledge graph-based big language model illusion detection method and device, and medium

Abstract

The invention discloses a big language model illusion detection method and device based on a knowledge graph and a medium, and relates to the technical field of natural language processing and knowledge engineering. The method comprises the steps of generating a to-be-detected text in response to a large language model, and obtaining a domain knowledge graph; performing structured analysis on the to-be-detected text to obtain an entity set and a relation triple corresponding to the to-be-detected text; based on the domain knowledge graph, performing factual illusion detection on the entity set and the relation triple to obtain a detection result and confidence corresponding to the factual illusion detection; and generating an illusion detection result corresponding to the to-be-detected text based on the detection result and the confidence coefficient. Therefore, the detection precision is high, the coverage range is wide, and the interpretability is high.

Description

Technical Field

[0001] This application relates to the fields of natural language processing and knowledge engineering technology, and in particular to a method, device and medium for detecting illusions based on a large language model using knowledge graphs. Background Technology

[0002] With the rapid development of large language models (such as the GPT series and LLaMA), the fluency and coherence of the generated text have reached a high level. However, the "illusion" problem severely restricts their application in high-reliability scenarios (such as medical consultations, legal documents, and news writing). They cannot meet the practical application requirements for detection accuracy, coverage, and interpretability. Summary of the Invention

[0003] This application provides a method, device, and medium for hallucination detection based on a knowledge graph and a large language model, in order to solve the following technical problem: how to improve the detection accuracy, coverage, and interpretability of hallucination detection.

[0004] In a first aspect, embodiments of this application provide a method for detecting illusions in a large language model based on a knowledge graph, comprising: generating a text to be detected in response to a large language model; obtaining a domain knowledge graph, wherein the domain corresponding to the domain knowledge graph is consistent with the domain corresponding to the text to be detected; performing structured parsing on the text to be detected to obtain an entity set and relation triples corresponding to the text to be detected; performing factual illusion detection on the entity set and relation triples based on the domain knowledge graph, obtaining a detection result and confidence level corresponding to the factual illusion detection, wherein the factual illusion detection includes: entity errors, relation errors, time discrepancies, and data fabrication; generating an illusion detection result corresponding to the text to be detected based on the detection result and the confidence level, wherein, when the illusion detection result indicates that the text to be detected is illusory, the illusion detection result further includes illusion type and error location.

[0005] Secondly, embodiments of this application also provide a knowledge graph-based large language model illusion detection device, the device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the knowledge graph-based large language model illusion detection method as described in the first aspect above.

[0006] Thirdly, embodiments of this application also provide a computer storage medium storing computer-executable instructions, which, when executed, implement the knowledge graph-based large language model hallucination detection method described in the first aspect above.

[0007] The hallucination detection method, device, and medium based on a knowledge graph and a large language model provided in this application have the following beneficial effects: In this embodiment, a domain-consistent domain knowledge graph can be obtained by generating the text to be detected in response to a large language model. Then, the text to be detected is subjected to structured parsing to obtain the entity set and relation triples corresponding to the text. Furthermore, based on the domain knowledge graph, factual illusion detection is performed on the entity set and relation triples to obtain the confidence level corresponding to the factual illusion detection. The factual illusion detection includes entity errors, relation errors, temporal inconsistencies, and data fabrication. This covers all illusion subcategories, making the detection dimensions more comprehensive. Moreover, using the domain knowledge graph to perform factual illusion detection on the entity set and relation triples corresponding to the text to be detected can improve the detection accuracy. Then, based on the detection results and confidence levels, an illusion detection result corresponding to the text to be detected can be generated. Where the illusion detection result indicates that the text to be detected contains illusions, the illusion detection result also includes the illusion type and error location. This makes the illusion detection result highly interpretable and suitable for high-reliability scenarios. Attached Figure Description

[0008] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A flowchart of a knowledge graph-based large language model hallucination detection method is provided for embodiments of this application; Figure 2 This is a schematic diagram of the internal structure of a knowledge graph-based large language model hallucination detection device provided in an embodiment of this application. Detailed Implementation

[0009] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0010] Existing Large Language Model (LLM) illusion detection technologies have significant shortcomings in high-reliability scenarios (such as medical, legal, and news writing), failing to meet the practical application requirements for detection accuracy, coverage, and interpretability. Specific deficiencies are as follows: 1. Low level of knowledge integration and insufficient utilization of structured information. Traditional hallucination detection relies on single-point queries of discrete knowledge bases, failing to fully exploit the structured relational features of knowledge. It can only verify isolated facts such as "whether an entity exists," unable to utilize the unique entity relationships, attribute hierarchy logic, and temporal topology of knowledge graphs. For example, for high-frequency hallucinations like "Einstein invented the telephone" (where the entity is correct but the relationship is incorrect), traditional methods are prone to missed detections due to a lack of structured verification capabilities at the relational level. Similarly, for temporal attribute errors like "the opening ceremony of the 2007 Beijing Summer Olympics," the lack of a structured comparison mechanism along the temporal dimension results in low detection accuracy.

[0011] 2. The detection dimension is limited, and the high-frequency hallucination subcategories are not fully covered. Existing technologies often focus on single illusion types such as "entity errors," failing to break down factual illusions into a complete sub-category system. They neglect high-frequency and high-risk illusion types such as "relationship errors," "time discrepancies," and "data fabrication" (e.g., data fabrication such as "a drug has a 70% recommendation rate" in the medical context, or time discrepancies such as "a contract's effective date is earlier than its signing date" in the legal context). Document experiment data shows that traditional methods (such as LLMcheck) have a detection rate of only 53.0% for "relationship errors" and cannot simultaneously cover all four core factual illusions, resulting in "one-sidedness" in the detection results.

[0012] 3. Lack of interpretability and engineering adaptability limits its practical value. Insufficient interpretability: Traditional methods only output a binary result of "illusion exists / no illusion exists", which cannot pinpoint the specific error location (such as a sentence, entity / relationship) or provide standard knowledge references. This results in a lack of clear basis for LLM iterative optimization, making it difficult for users to correct errors.

[0013] Poor model adaptability: The detection process needs to be redesigned for different LLM architectures (such as Qwen-3 and LLaMA 3), and compatibility cannot be achieved through lightweight parameter adjustments. Qwen-3 is a new generation of open-source large language models released by Alibaba's Qwen Team on April 29, 2025. LLaMA 3 (Large Language Model Meta AI3) is the third generation of open-source large language models released by Meta Labs on April 18, 2024.

[0014] Lagging knowledge timeliness: The lack of a dynamic update mechanism makes it impossible to accurately detect time-sensitive facts such as "event time updates" and "statistical data iterations" (e.g., the time of clauses after policy revisions), further limiting its application in real-time scenarios.

[0015] This application provides a method for detecting hallucinations using a large language model based on knowledge graphs. The technical solution proposed in this application will be described in detail below with reference to the accompanying drawings.

[0016] Figure 1 This document provides a flowchart of a knowledge graph-based large language model hallucination detection method as an embodiment of this application. Figure 1 As shown in the figure, the hallucination detection method based on a knowledge graph and a large language model provided in this application specifically includes the following steps: Step 101: Respond to the large language model to generate the text to be detected and obtain the domain knowledge graph.

[0017] The domain corresponding to the domain knowledge graph is consistent with the domain corresponding to the text to be detected.

[0018] In this embodiment, the type of large language model is not limited; it can be a general-purpose large language model or a domain-specific large language model. After the large language model generates the text to be detected based on the user's input, a domain knowledge graph consistent with the domain of the text to be detected can be obtained. A domain-specific knowledge graph (DKG) is a structured knowledge system built for a specific industry or professional field. The domain knowledge graph constructs a semantic network through entity-relationship-attribute triples, combines an ontology-standard logical structure, and relies on graph databases / RDF storage to achieve efficient querying, ultimately serving a specific industry. Its core lies in transforming unstructured data into structured knowledge, which can support deep reasoning and decision support. Core elements include: entities, relations, and attributes. Entities represent concrete objects or abstract concepts in the real world, such as people, places, organizations, products, and events. Relationships describe the semantic associations between entities, such as "person-born-place," "company-belongs to-industry," and "drug-treatment-disease." Relationships typically possess directionality (e.g., "A→B" represents a relationship from A to B) and type (e.g., "parent-child," "employment"). Attributes can be used to describe the characteristics or state of an entity, such as "age," "coordinates," or "establishment date." Attribute values ​​can be text, numerical, time, or geographic coordinates. Thus, when performing hallucination detection on text to be tested, leveraging domain knowledge graphs can improve the degree of knowledge fusion and the utilization rate of structured information, thereby helping to improve the accuracy of hallucination detection on the text to be tested, especially suitable for high-reliability text generation scenarios such as medical and legal fields.

[0019] Step 102: Perform structured parsing on the text to be detected to obtain the entity set and relation triplet corresponding to the text to be detected.

[0020] In practical applications, structured parsing refers to the process of transforming unstructured or semi-structured raw data (such as natural language text, images, audio, etc.) into machine-readable, logically clear, and uniformly formatted structured information. The core objective is to achieve standardized data representation and knowledge extraction. Therefore, in this embodiment, the text to be detected (which can be denoted as G) can be structured parsed to obtain the entity set E corresponding to the text to be detected. G and relation triple T G In this framework, entities can refer to concrete or abstract things such as people, places, and events in the text to be detected, while relation triples are in the form of (entity 1, relation, entity 2), which can represent the relationship between two entities in the text to be detected. In this way, unstructured text to be detected can be transformed into machine-understandable structured knowledge, which facilitates rapid hallucination detection. Moreover, the results of structured parsing help to clearly mark the specific location of hallucination information (e.g., "entity 'X' does not exist" or "relation 'Y' contradicts the facts"), making the detection results more interpretable and facilitating subsequent review and correction.

[0021] Step 103: Based on the domain knowledge graph, perform factual illusion detection on the entity set and the relation triples to obtain the detection results and confidence levels corresponding to the factual illusion detection.

[0022] The factual hallucination detection includes: entity errors, relational errors, temporal distortions, and data fabrication.

[0023] In practical applications, factual hallucination refers to a contradiction between the content generated by a large language model and verifiable facts. Examples include fabricating non-existent historical events (such as "Einstein discovered general relativity in 1905"), incorrect timing, fictitious data, or logical errors (such as incorrect medical diagnoses or fabricated legal precedents). Factual hallucination detection is a technical system for identifying and verifying phenomena in the content generated by a large language model that are "inconsistent with reality" or "lacking evidence." Specifically, it can include entity errors, relational errors, temporal inconsistencies, and data fabrication. Existing technologies generally focus on single hallucination types such as "entity errors," without breaking down factual hallucination into a complete sub-category system. However, in this embodiment, the aforementioned domain knowledge graph can be used to perform factual hallucination detection on the entity set and relation triples corresponding to the text to be detected, obtaining the confidence level corresponding to the factual hallucination detection. This approach covers all illusion subcategories, resulting in a more comprehensive detection dimension. Furthermore, employing the aforementioned domain knowledge graph to perform factual illusion detection on the entity set and relation triples corresponding to the text to be detected improves detection accuracy. In practical applications, confidence level is a core concept in statistics and machine learning, used to quantify the reliability of estimated or predicted results. When obtaining the detection results for factual illusion detection, a confidence level assessment can be performed on these results, and then mapped to a probability range of 0-1 to represent the reliability of the results. This quantifies uncertainty and supports subsequent decision-making.

[0024] Step 104: Based on the detection results and the confidence level, generate the hallucination detection results corresponding to the text to be detected.

[0025] Wherein, if the hallucination detection result indicates that the text to be detected is hallucinating, the hallucination detection result also includes the hallucination type and the error location.

[0026] In this embodiment, the final hallucination detection result can be obtained based on the above detection results and their confidence level. In practical applications, the hallucination detection result can also include standard or correct information corresponding to the hallucination provided by the domain knowledge graph, facilitating subsequent correction of the text to be detected. Thus, the hallucination detection result not only includes the hallucination type but also outputs the error location, providing a clear basis for the iterative optimization of LLM and exhibiting strong interpretability.

[0027] In this embodiment, the system can generate a text to be detected in response to a large language model, and obtain a domain-consistent knowledge graph. Then, it performs structured parsing on the text to be detected to obtain the entity set and relation triples corresponding to the text. Furthermore, based on the domain knowledge graph, it performs factual illusion detection on the entity set and relation triples, obtaining the detection results and confidence levels. Factual illusion detection includes entity errors, relation errors, temporal inconsistencies, and data fabrication. This covers all illusion subcategories, making the detection more comprehensive. Moreover, using the domain knowledge graph to perform factual illusion detection on the entity set and relation triples corresponding to the text to be detected improves detection accuracy. Finally, based on the detection results and confidence levels, it can generate illusion detection results for the text to be detected. When the illusion detection results indicate that the text to be detected contains illusions, the illusion detection results also include the illusion type and error location. This makes the illusion detection results highly interpretable and suitable for high-reliability scenarios.

[0028] In practical applications, a domain knowledge graph for a specific domain can only be used for illusion detection of the text to be detected within that domain. Therefore, to improve detection accuracy, before acquiring the domain knowledge graph, it can be checked whether the domain corresponding to the text or the text to be detected is consistent with the domain corresponding to the domain knowledge graph. If the two domains are inconsistent, illusion detection can be omitted. The method provided in the embodiments of this application can be omitted from the illusion detection process; other methods can be used instead. Alternatively, illusion detection can be omitted entirely; there are no specific limitations. This ensures that illusion detection is performed on the text to be detected within a specific domain, saving resources and time.

[0029] In one possible implementation, prior to acquiring the domain knowledge graph, the method further includes: Extract relation triples and entity attributes from the data source to form an initial knowledge graph; An entity linking algorithm based on contextual semantic similarity is used to disambiguate entities in the initial knowledge graph to solve the problem of confusion between entities with the same name; The missing entity relationships in the initial knowledge graph are completed by using a knowledge graph embedding model; Add structured tags to entities or relationships involving time or data to obtain an optimized domain knowledge graph.

[0030] In practical applications, the domain knowledge graph is typically determined based on user-provided requirements. For example, if a user wants to perform hallucination detection on text in the medical field, a medical domain knowledge graph needs to be constructed. Furthermore, the subsequent text to be detected will also be in the medical field (in certain scenarios, regardless of the type of large language model, the input text to the model is generally in the medical field, and the generated text is also in the medical field). In the above embodiment, a domain knowledge graph can be constructed and optimized before performing hallucination detection. This improves the efficiency of hallucination detection, avoids wasting time by temporarily constructing the domain knowledge graph after the large oracle model generates the text to be detected, and helps improve user satisfaction.

[0031] In the above embodiments, triples and entity attributes can be extracted from the data source to form an initial knowledge graph. The data source can be a public database, such as Wikipedia, DBpedia, or a domain-specific industry database. Wikipedia is a multilingual, collaborative, and free online encyclopedia. DBpedia is a semantic web core knowledge base that extracts structured data from Wikipedia and can provide open knowledge graph services covering multiple domains such as people, places, and organizations. A domain-specific industry database refers to a database system designed for a specific industry or professional field, specifically used to collect, organize, and store structured data and information resources for that industry; for example, a database in the medical field. In practical applications, information from the above data sources can be integrated to extract relation triples <entity E1, relation R, entity E2> and entity attributes (such as time attributes and numerical attributes) to form an initial knowledge graph.

[0032] After constructing the initial knowledge graph, it can be optimized. Entity linking algorithms based on contextual semantic similarity can be used to disambiguate entities in the initial knowledge graph, resolving the problem of confusion between entities with the same name (e.g., "apple" could refer to a fruit or a company). Furthermore, Knowledge Graph Embedding (KGE) models can be used to complete the missing entity relationships in the initial knowledge graph, improving the detection of "relationship error" illusions. In practical applications, the knowledge graph embedding model can be a TransE (Translation-based Embedding) model, whose core idea is to map entities and relationships in the knowledge graph to a low-dimensional continuous vector space, capturing semantic associations through "vector translation" operations. The specific type of KGE model is not limited. Standardization can also be applied to time or numerical attributes, adding structured labels (such as start_time and end_time for events, and source and range for statistical data) to entities or relationships involving time or data to detect "time discrepancies" and "data fabrication." Through these methods, an optimized domain knowledge graph can be obtained. In this way, by reducing noise through disambiguation, enhancing completeness through completion, and injecting dynamic information through annotation, the semantic understanding ability of the knowledge graph is systematically enhanced, making the domain knowledge graph closer to the complexity and dynamism of the real world, thereby improving the accuracy of hallucination detection in the text to be detected.

[0033] In practical applications, the domain knowledge graph can be updated periodically. For example, it can support the regular access to the latest knowledge base (such as monthly updates from Wikipedia) and update the entities, relationships, and attributes of the DKG through incremental learning to ensure the accuracy of detecting time-sensitive facts (such as event times and statistical data). This can improve the knowledge expiration rate of hallucination detection and enhance the application capability of the above methods in real-time scenarios.

[0034] In one possible implementation, the step of performing structured parsing on the text to be detected to obtain the entity set and relation triplet corresponding to the text to be detected includes: A named entity recognition model is used to extract the set of entities from the text to be detected. The relation triples in the text to be detected are extracted using a relation extraction model, and the time information, numerical data and logical connectives in the text to be detected are also extracted.

[0035] In practical applications, a Named Entity Recognition (NER) model can be used to extract the entity set E from G. G = {e1, e2, ..., e nThe entity types (such as people, locations, events) are labeled. The aforementioned NER model can be a BERT-based NER model, which is an entity recognition system built using Bidirectional Encoder Representations from Transformers (BERT) as its core, through pre-training and fine-tuning paradigms. It can also be a cross-lingual and multimodal model; there are no specific restrictions. Using the NER model, the entity set in the text to be detected can be quickly extracted. Furthermore, after entity extraction via NER, the output can be standardized, facilitating subsequent relation and attribute extraction to obtain relation triples.

[0036] Then, relation and attribute extraction can be performed. Entity relation triples T in G can be extracted using a relation extraction model (such as RE-BERT). G = { <e i , r j , e k The algorithm extracts time information (e.g., "built in the 18th century"), numerical data (e.g., "70% of doctors"), and logical connectors (e.g., "because...therefore...", "firstly...secondly...") from the text. RE-BERT, short for Relation Extraction BERT, is a customized variant of BERT for relation classification tasks, improving relation recognition accuracy through entity location labeling and feature fusion mechanisms.

[0037] It should be noted that in practical applications, the parameters of the above-mentioned relationship extraction model can be adjusted according to the output style relationships of the large language model without changing the overall architecture. In this way, for large language models with different architectures (such as Qwen-3 and LLaMA 3), there is no need to redesign the detection process; compatibility can be achieved through lightweight parameter adjustments.

[0038] In one possible implementation, when the factual illusion detection includes entity errors, i.e., when entity error detection of the text to be detected is required, the detection can be performed based on a domain knowledge graph.

[0039] The method of performing factual illusion detection on the entity set and the relation triples based on the domain knowledge graph includes: The entities in the entity set are matched with the entity library of the domain knowledge graph, and the entities that are not matched are marked as candidate erroneous entities; For the matched entities, extract the first set of attributes of the entities in the text to be detected; The first set of attributes is compared with the second set of attributes corresponding to the entity in the domain knowledge graph, and the attribute similarity is calculated. If the attribute similarity is less than a first preset threshold, it is determined to be an entity error.

[0040] In practical applications, when performing entity error detection, we can first verify the existence of the entity, and then... G Each entity e in i Match with the entity library of DKG, if e i If there is no corresponding node in the DKG, it is marked as a candidate for "entity error".

[0041] Then perform entity attribute consistency verification. If e i It exists in DKG, and the description of e can be extracted from G. i Attributes (such as "The Eiffel Tower was built in the 18th century"), and e in DKG i The attributes (such as "construction date: 1889") are compared, and the attribute similarity is calculated:

[0042] Among them, A G (e i ) is G in e i The set of attributes (the first set of attributes), A DKG (e i Let be the set of attributes (second attribute set) of ei in DKG, and I(·) be the indicator function. When Sim attr (e i When ) < θ1 (θ1 is the first preset threshold), it can be determined as "entity error".

[0043] The above methods can fully utilize structured information such as entity relationships in the domain knowledge graph, significantly improving the accuracy of entity error detection. In practical applications, other methods can also be used to perform entity error detection on the text to be detected based on the domain knowledge graph; no specific limitations are imposed.

[0044] In one possible implementation, when the factual illusion detection includes relation errors, the factual illusion detection based on the domain knowledge graph for the entity set and the relation triples includes: Match the relation triples with the relation triples in the domain knowledge graph; If the entities match but the relationships do not, mark it directly as a relationship error; When the entities in the relation triple do not have corresponding relations in the domain knowledge graph, the similarity between the relations in the relation triple and the potential relations between the entities in the domain knowledge graph is calculated using a knowledge graph embedding model. If the similarity is less than a second preset threshold, the relationship is determined to be incorrect.

[0045] In practical applications, relation error detection primarily involves relation triple matching, which can be performed on triples in the relation trigonometric identity (TG). <e i , r j , e k >Triples in DKG<E1, R, E2> Matching can be performed in two ways: (1) If e i Corresponding to E1, e k Corresponding to E2, but r j If there is a mismatch with R (e.g., "Einstein invented the telephone" in G corresponds to "Bell invented the telephone" in DKG), it can be directly marked as "relation error".

[0046] (2) If e i With e k There is no r in DKG j The corresponding relationship (e.g., "Beijing is the capital of France") is calculated using a knowledge graph embedding model, and r is used to calculate the value. j With DKG e i e k If the similarity between potential relationships is lower than θ2 (the second preset threshold), it can be determined as "relationship error".

[0047] The above method fully utilizes the structured information of the domain knowledge graph, significantly improving the accuracy of relation error detection. In practical applications, other methods can also be used to detect relation errors in the text based on the domain knowledge graph; no specific limitations are imposed.

[0048] In one possible implementation, when the factual illusion detection includes temporal distortion, the factual illusion detection based on the domain knowledge graph for the entity set and the relation triples includes: The time information extracted from the text to be detected is converted into a timestamp format and associated with the corresponding event entity; Query the standard time attributes of the time entities in the domain knowledge graph; Calculate the time difference between the timestamp format and the standard time attribute; If the time difference exceeds a preset time threshold or there is a time logic conflict, it is determined to be a time disorder.

[0049] In practical applications, when detecting time discrepancies, the time information can first be structured. The time information extracted from G (such as "the opening ceremony of the 2007 Summer Olympics held in Beijing") can be converted into a timestamp format T. G The time (e.g., "2007-01-01") is associated with the corresponding event entity e. event .

[0050] Then perform time topology verification, querying e in DKG. event Standard Time Attribute T DKG Time (e.g., "Opening ceremony of the Beijing Summer Olympics: 2008-08-08"), calculate the time difference ΔT = |T G time - T DKG time|. If ΔT exceeds a preset time threshold (e.g., 1 year), or T G If the timeline conflicts with the time logic of events in the DKG (such as "World War I broke out after World War II"), it is judged as "time disorder".

[0051] The above methods can fully utilize structured information such as the temporal topology of the domain knowledge graph, significantly improving the detection accuracy of temporal discrepancies. In practical applications, other methods can also be used to detect temporal discrepancies in the text based on the domain knowledge graph; there are no specific limitations.

[0052] In one possible implementation, where the factual illusion detection includes data fabrication, the factual illusion detection based on the domain knowledge graph of the entity set and the relation triples includes: If the source of the numerical data in the text to be detected is not labeled, query the reference data in the corresponding field in the DKG; Calculate the data deviation based on the numerical data and reference data; If the data deviation is greater than a third preset threshold, it is determined to be data fabrication.

[0053] In practical applications, when detecting data fabrication, one can first verify the data source and extract the statistical data D from G. G (e.g., "70% of doctors worldwide recommend a certain drug"), check if the data source is indicated in G. If the source is not indicated, proceed to the next step of data range and reasonableness verification, and search for reference data D in the corresponding field in DKG. DKG (e.g., "Reference range for doctor's recommendation rate of a certain drug: 30%-50%)", calculate data deviation:

[0054] When Bias(D)G When the threshold value is greater than θ3 (θ3 is the third preset threshold), it can be determined as "data fabrication".

[0055] The above methods can fully utilize the reference data of the domain knowledge graph, significantly improving the accuracy of data fabrication detection. In practical applications, other methods can also be used to detect data fabrication in the text to be tested based on the domain knowledge graph; there are no specific limitations.

[0056] In practical applications, the initial values ​​of θ1 to θ3 and the preset time threshold can be determined by performing five-fold cross-validation on public illusion datasets (such as FaithDial and HaluEval). In specific practices, these values ​​can be adaptively adjusted according to the characteristics of the domain data. For example, in the medical field, the requirements for the text to be detected are higher. In this case, compared with other fields, it is necessary to lower the first preset threshold to minimize the probability of errors in drug names, etc.

[0057] In one possible implementation, generating the hallucination detection result corresponding to the text to be detected based on the detection result and the confidence level includes: A fused feature vector is formed by concatenating the confidence scores corresponding to the aforementioned factual hallucination detections; A classifier is used to classify the fused feature vector, and based on the detection results, the hallucination detection results corresponding to the text to be detected are obtained.

[0058] In the above embodiments, the confidence level corresponding to the detection of factual hallucinations (e.g., entity error confidence level Conf) can be used to measure the confidence level corresponding to the detection of factual hallucinations. fact1 Relationship Error Confidence Conf fact2 The features are concatenated to construct a fused feature vector F. At this point, F = [Conf]. fact1 Conf fact2 Different confidence levels reflect the reliability of errors in the existence of different categories of factual illusions, such as entity errors and relational errors. Concatenating these can form a more comprehensive feature representation, thereby more accurately determining whether a hallucination exists, and also avoiding the omission of certain hallucinations.

[0059] In practical applications, a classifier can be used to classify the aforementioned fused feature vectors to obtain the hallucination detection results for the text to be detected. For example, a lightweight classifier (such as logistic regression) can be used to classify F, outputting the final hallucination detection result. If the confidence score of any subclass exceeds the corresponding threshold, the text to be detected, G, can be determined to be "illusory," and based on the detection result, the hallucination type (e.g., "relationship error," "time discrepancy") and specific location (e.g., sentence number, incorrect entity / relationship) can be output. If the confidence scores of all subclasses are below the corresponding threshold, the text to be detected, G, can be determined to be "no hallucination." Thus, if it is determined that the text to be detected does not contain hallucinations, the hallucination detection result is "no hallucination." In this way, the hallucination detection result not only includes the hallucination type but also outputs the error location, providing a clear basis for the iterative optimization of LLM and exhibiting strong interpretability.

[0060] To make the above technical solution clearer, the method is explained in detail below with specific detection examples. This embodiment uses the detection of text G generated by a large language model as an example. test For example, G test The content reads: "In the 18th century, Einstein invented the telephone, a technology recommended by 70% of doctors worldwide for telemedicine." (1) Constructing a domain knowledge graph. This embodiment uses a domain knowledge graph (DKG). test Built based on the following data sources: General knowledge base: Integrates core triples from DBpedia and Wikipedia, such as <Albert Einstein, occupation, physicist>, <telephone, inventor, Alexander Graham Bell>, <telephone, invention year, 1876>.

[0061] Domain Database: Reference data on "Recommendation Rate of Remote Diagnosis and Treatment Technology" in the medical industry database, ranging from 25% to 40%, with an average of 32%.

[0062] Optimized results: The relationship <remote diagnosis, core tools, telephone> was completed using the TransE model; entity disambiguation confirmed that "Einstein" corresponds to the physicist entity, and other entities with the same name were excluded.

[0063] (2) Generating text parsing results. Using BERT-base-chinese as the NER model and RE-BERT as the relation extraction model, the parsing results for Gtest are as follows: Entity set: E G = {Einstein, telephone, 18th century, 70% of doctors worldwide, telemedicine} relational triples: T G= {<Einstein, invented the telephone>,<the telephone is recommended for, telemedicine>,<70% of doctors, recommend, the telephone>} Time / numerical information: T G time = {Telephone invention time: 18th century}; D G = {70% (doctor's recommendation rate)} (3) ① Entity error detection Match each entity in E_G to the DKG. test "Einstein", "telephone", and "telemedicine" all have entity nodes; "18th century" and "70% of doctors worldwide" are attribute / data descriptions, not entities, and do not require existence verification.

[0064] Entity attribute validation: Extracting G test The Chinese text "Telephone Invention Time: 18th Century" is related to DKG. test The similarity score Sim was calculated by comparing the date of invention of the telephone (1876, 19th century) with the date of invention of the telephone. attr (Telephone) = 0 (due to a complete mismatch in time attributes). Since 0 < θ1 = 0.6 (preset threshold), the time attribute description for "Telephone" is marked as an incorrect candidate for entity attribute.

[0065] ② Relationship error detection (core subcategory) Match the triple <Einstein, Invention, Telephone> in T_G with DKG test DKG test The corresponding triple is <Alexander Bell, Invention, Telephone>.

[0066] Calculate the relationship similarity: The cosine similarity between the embedding vector of "Einstein-invention-telephone" and the embedding vector of "Bell-invention-telephone" calculated by the TransE model is 0.12, which is lower than θ2=0.5 (preset threshold), so it is judged as a relationship error.

[0067] ③ Time discrepancy detection Time-structured transformation: "18th century" is transformed into a time range [1701-1800], associated with the event entity "invention of the telephone".

[0068] Time topology verification: DKG test The text states that "telephone was invented in 1876 (1801-1900, 19th century)". The calculated time difference ΔT = 1876 - 1750 (mid-18th century) = 126 years, which exceeds the preset time threshold (50 years), and can be judged as a time discrepancy.

[0069] ④ Data fabrication detection Data source verification: G testThe claim that "70% of doctors recommend it" lacks source information and is subject to a validity check.

[0070] Calculate data deviation: D G =70%, DKG test Chinese reference data D DKG ^{avg}=32%, D DKG ^{max}=40%, D DKG ^{min}=25%, substitute into the formula:

[0071] Since 2.53 > θ3 = 0.8 (preset threshold), it is determined to be data fabrication.

[0072] (4) Perform feature fusion and obtain hallucination detection results.

[0073] Construction of fused feature vectors: The confidence scores (based on model output probabilities) for each subcategory are: Conf entity error = 0.92, Conf relationship error = 0.98, Conf time disorder = 0.95, Conf data fabrication = 0.91, and the fused feature vector F = [0.92, 0.98, 0.95, 0.91].

[0074] Final decision: The logistic regression classifier judges F. If the confidence score of all subclasses exceeds the corresponding threshold (threshold range 0.7-0.8), the detection result is output: Illusion types: Relationship error (core), Time distortion, Data fabrication, Entity error (attribute) Incorrect entries: "Einstein invented the telephone" (incorrect relationship + chronological error), "70% of doctors worldwide recommend it" (fabricated data).

[0075] Recommended fix: Based on DKG test Provide standard information such as "the telephone was invented by Bell in 1876" and "the telephone recommendation rate in telemedicine is about 32%".

[0076] (5) Implementation effect verification This embodiment verifies the effect through comparative experiments, selecting a traditional knowledge-based detection method (Baseline) and comparing it with the above method to detect G. test The generated text, along with 100 texts containing mixed hallucinations, is shown in the table below:

[0077] Experimental results show that the above method can significantly improve detection accuracy while covering all hallucination subcategories, especially for high-frequency hallucinations such as "relationship error", without increasing computational overhead.

[0078] The above are embodiments of the method proposed in this application. Based on the same inventive concept, embodiments of this application also provide a knowledge graph-based hallucination detection model, which can combine multi-dimensional feature fusion and decision-making mechanisms to achieve comprehensive hallucination recognition of generated text. The core architecture includes: a knowledge graph preprocessing module, a generated text parsing module, a factual hallucination detection module, and a feature fusion and decision-making module.

[0079] The knowledge graph preprocessing module is used to construct and optimize the domain knowledge graph (DKG) for hallucination detection. The text parsing module performs structured parsing on the text to be detected (G) generated by the LLM, outputting the core elements (entity set E) required for detection. G and relation triple T G The factual hallucination detection module is used to analyze the parsed E based on the domain knowledge graph (DKG). G and T G Element-by-element verification is performed, and the detection results and confidence scores for each subcategory of hallucination are output. The feature fusion and decision module is used to output the hallucination detection results based on the aforementioned category confidence scores.

[0080] Based on the same inventive concept, this application also provides a knowledge graph-based large language model hallucination detection device, the structure of which is as follows: Figure 2 As shown.

[0081] Figure 2 This is a schematic diagram of the internal structure of a knowledge graph-based large language model hallucination detection device provided in an embodiment of this application. Figure 2 As shown, the device includes: At least one processor 201; And a memory 202 that is communicatively connected to at least one processor; The memory 202 stores instructions that can be executed by at least one processor. The instructions are executed by at least one processor 201 to enable at least one processor 201 to: execute the knowledge graph-based large language model illusion detection method described above.

[0082] Some embodiments of this application provide corresponding to Figure 1 A non-volatile computer storage medium stores computer-executable instructions, which are configured to execute the aforementioned knowledge graph-based large language model illusion detection method.

[0083] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments for IoT devices and media are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0084] The systems, media, and methods provided in this application are one-to-one correspondences. Therefore, the systems and media also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the systems and media will not be repeated here.

[0085] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0086] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0087] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0088] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0089] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0090] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0091] Computer-readable media include both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0092] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0093] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for hallucination detection based on a large language model using knowledge graphs, characterized in that, include: In response to the generation of the text to be detected by the large language model, a domain knowledge graph is obtained, wherein the domain corresponding to the domain knowledge graph is consistent with the domain corresponding to the text to be detected; The text to be detected is subjected to structured parsing to obtain the entity set and relation triplet corresponding to the text to be detected; Based on the domain knowledge graph, factual illusion detection is performed on the entity set and the relation triples to obtain the detection results and confidence levels corresponding to the factual illusion detection. The factual illusion detection includes: entity error, relation error, time disorder, and data fabrication. Based on the detection results and the confidence level, a hallucination detection result corresponding to the text to be detected is generated. Wherein, if the hallucination detection result indicates that the text to be detected is hallucinating, the hallucination detection result also includes the hallucination type and the error location.

2. The method according to claim 1, characterized in that, Prior to acquiring the domain knowledge graph, the method further includes: Extract relation triples and entity attributes from the data source to form an initial knowledge graph; An entity linking algorithm based on contextual semantic similarity is used to disambiguate entities in the initial knowledge graph to solve the problem of confusion between entities with the same name; The missing entity relationships in the initial knowledge graph are supplemented by a knowledge graph embedding model; Add structured tags to entities or relationships involving time or data to obtain an optimized domain knowledge graph.

3. The method according to claim 1, characterized in that, The step of performing structured parsing on the text to be detected to obtain the entity set and relation triplet corresponding to the text to be detected includes: A named entity recognition model is used to extract the set of entities from the text to be detected. The relation triples in the text to be detected are extracted using the relation extraction model, and the time information, numerical data and logical connectors in the text to be detected are also extracted.

4. The method according to claim 1, characterized in that, In cases where the factual illusion detection includes relation errors, the step of performing factual illusion detection on the entity set and the relation triples based on the domain knowledge graph includes: The entities in the entity set are matched with the entity library of the domain knowledge graph, and the entities that are not matched are marked as candidate erroneous entities; For the matched entities, extract the first set of attributes of the entities in the text to be detected; The first set of attributes is compared with the second set of attributes corresponding to the entity in the domain knowledge graph, and the attribute similarity is calculated. If the attribute similarity is less than a first preset threshold, it is determined to be an entity error.

5. The method according to claim 1, characterized in that, In the case of factual illusion detection involving temporal distortion, the step of performing factual illusion detection on the entity set and the relation triples based on the domain knowledge graph includes: Match the relation triples with the relation triples in the domain knowledge graph; If the entities match but the relationships do not, mark it directly as a relationship error; When the entities in the relation triple do not have corresponding relations in the domain knowledge graph, the similarity between the relations in the relation triple and the potential relations between the entities in the domain knowledge graph is calculated using a knowledge graph embedding model. If the similarity is less than a second preset threshold, the relationship is determined to be incorrect.

6. The method according to claim 3, characterized in that, In cases where the factual illusion detection includes data fabrication, the step of performing factual illusion detection on the entity set and the relation triples based on the domain knowledge graph includes: The time information extracted from the text to be detected is converted into a timestamp format and associated with the corresponding event entity; Query the standard time attributes of the time entities in the domain knowledge graph; Calculate the time difference between the timestamp format and the standard time attribute; If the time difference exceeds a preset time threshold or there is a time logic conflict, it is determined to be a time disorder.

7. The method according to claim 3, characterized in that, In cases where the factual illusion detection includes entity errors, the step of performing factual illusion detection on the entity set and the relation triples based on the domain knowledge graph includes: If the source of the numerical data in the text to be detected is not labeled, query the reference data in the corresponding field in the DKG; Calculate the data deviation based on the numerical data and reference data; If the data deviation is greater than a third preset threshold, it is determined to be data fabrication.

8. The method according to claim 1, characterized in that, The step of generating a hallucination detection result corresponding to the text to be detected based on the detection result and the confidence level includes: A fused feature vector is formed by concatenating the confidence scores corresponding to the aforementioned factual hallucination detections; A classifier is used to classify the fused feature vector, and based on the detection results, the hallucination detection results corresponding to the text to be detected are obtained.

9. A knowledge graph-based large language model hallucination detection device, characterized in that, The device includes: At least one processor; And, a memory communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform a knowledge graph-based large language model illusion detection method as described in any one of claims 1-8.

10. A computer storage medium storing computer-executable instructions, characterized in that, When the computer-executable instructions are executed, they implement the large language model illusion detection method based on knowledge graphs as described in any one of claims 1-8.

Images