Large language model illusion detection method and system based on knowledge graph structure

Abstract

The invention discloses a big language model illusion detection method and system based on a knowledge graph structure, and belongs to the field of big model illusion detection. The method comprises the following steps: acquiring text data generated by a large language model, and extracting a triple of the text data to obtain a knowledge graph triple representation set; on the basis of a large language model and a natural language inference model, whether triads in the knowledge graph triad representation set have illusion or not is judged, and results of the two models are output as final illusion probabilities; and outputting the illusion existence probability of each triple, and performing visual display. The double-model verification mechanism provided by the invention improves the reliability of the hallucination detection result, the weighted fusion algorithm can balance the advantages of different models, the position of the hallucination is accurately positioned, and visual display is carried out to facilitate the understanding of a user.

Description

Technical Field

[0001] The present invention relates to the field of large model hallucination detection, and in particular to a large language model hallucination detection method and system based on a knowledge graph structure. Background Art

[0002] With the rapid development of Large Language Model (LLM) technology, pre-trained language models represented by GPT and DeepSeek have achieved remarkable results in the field of natural language processing. However, when generating text, such models often produce content that is inconsistent with the facts or given context, the so-called "hallucination", which seriously affects the reliability and practicality of the model output.

[0003] In the prior art, the patent "Large Model Hallucination Detection Method, Device, Computer Equipment and Storage Medium" mentions: After obtaining the text data generated by the large model, parse the text data to obtain standard events; generate query statements corresponding to the standard events, execute the query statements, and obtain query results; verify the standard events according to the query results to obtain the large model hallucination detection results. This method is used to perform text parsing on the text data generated by the large model, identify the content that may contain hallucinations, and make the hallucination detection targeted; parse the content that may contain hallucinations into standard events, eliminate the differences between text data with different expressions of the same meaning, and enhance data comparability; verify the authenticity of the standard events based on the real data obtained by the query, and can detect events that do not conform to the facts in the text data generated by the large model, thereby realizing the detection of hallucinations of the large model.

[0004] However, this patent requires hallucination testing of parsed standard events based on query results, and does not involve hallucination detection to determine whether the text content generated by the large model conforms to the given context.

[0005] The paper "Identification and Optimization of Hallucination Phenomena in Large Language Models" mentioned: Based on the public dataset Huatuo, a large model hallucination evaluation dataset in the field of medical question and answer was constructed by combining GPT4 to generate question answers and manual annotations; secondly, based on the constructed hallucination evaluation dataset, the concept of "hallucination rate" was defined, and the degree of hallucination of each large model was tested and quantified by designing prompts to let the model to be tested answer "yes" or "no".

[0006] However, this paper only guides the big model to output the answer to the question while also outputting a judgment on whether the answer contains hallucinations. It does not detect the presence of hallucination units in the output of the big model, nor does it involve hallucination detection to determine whether the text content generated by the big model conforms to the given context.

[0007] Existing methods mostly rely on simple text similarity matching or keyword retrieval, which cannot deeply understand the consistency of text semantics. The lack of a systematic and structured detection framework makes it difficult to accurately locate the specific location of hallucination content. The detection results lack interpretability, making it difficult to show users the causes and evidence of hallucinations. Summary of the invention

[0008] In view of the deficiencies in the prior art, the object of the present invention is to provide a large language model hallucination detection method and system based on a knowledge graph structure.

[0009] The object of the present invention is to achieve the following technical solution: a large language model hallucination detection method based on a knowledge graph structure, comprising:

[0010] Obtaining text data output by the first language model, generating triples in RDF format according to the text data; supplementing the triples of the text data through multiple rounds of interaction to obtain a knowledge graph triple representation set;

[0011] Based on the second largest language model, the consistency between each triple in the knowledge graph triple representation set and the corresponding text context is determined, and a first hallucination probability is calculated;

[0012] Based on the natural language inference model, the consistency between each triple in the knowledge graph triple representation set and the corresponding text context is determined, and the second hallucination probability is calculated;

[0013] According to the first hallucination probability and the second hallucination probability, a final hallucination probability is obtained to determine whether the hallucination of the same triple is true;

[0014] Output the final hallucination probability of each triplet.

[0015] Furthermore, the step of obtaining text data output by the first language model and generating triples in RDF format according to the text data includes:

[0016] The first language model is guided by prompt to extract entities and their relationships from unstructured natural language text, and constraints are set to generate triples that conform to the RDF format;

[0017] The generated triples are verified and post-processed, and the erroneous triples are corrected.

[0018] Furthermore, extracting entities and their relations includes providing examples in prompts to help the model understand the task.

[0019] Furthermore, the triples of text data are supplemented through multiple rounds of interaction to obtain a knowledge graph triple representation set including:

[0020] After the triples are generated for the first time, the triples and the corresponding texts are input into the first largest language model to determine whether there are any unextracted triples. If so, they are supplemented and the supplemented triples are verified and corrected to obtain a set of triple representations of the completed knowledge graph.

[0021] Furthermore, judging the consistency between each triple in the knowledge graph triple representation set and the corresponding text context based on the second largest language model and calculating the first hallucination probability includes:

[0022] Setting a prompt word template, including a triple and a corresponding text context, wherein the text context includes a head entity, a relation, and a tail entity;

[0023] The prompt word is input into the second largest language model, the consistency of the triple is judged by the text context, and the output result is mapped to the first hallucination probability; if correct, the result is mapped to a probability value of 0, if wrong, the result is mapped to a probability value of 1, and if partially correct, the result is mapped to a probability value between 0 and 1 according to the part corresponding to the triplet and the text context.

[0024] Furthermore, judging the consistency between each triple in the knowledge graph triple representation set and the corresponding text context based on the natural language inference model and calculating the second hallucination probability includes:

[0025] Convert the triple into a natural language statement as a hypothesis H. When the sum of the length of the text context corresponding to the triple and the length of the hypothesis H does not exceed the maximum input length of the natural language inference model, use the text context directly as the premise P. When the sum of the length of the text context and the length of the hypothesis H exceeds the maximum input length of the natural language inference model, extract context information related to the triple from the text context as the premise P.

[0026] The hypothesis H and premise P are input into a natural language reasoning model, and the logical relationship between the hypothesis H and the premise P is inferred through the natural language reasoning model, and the probability distribution of three relationships: implication, contradiction and neutrality is output; the contradiction probability output by the model is used as the second hallucination probability of the triple.

[0027] Further, obtaining a final hallucination probability according to the first hallucination probability and the second hallucination probability, and judging whether the hallucination of the same triple is true comprises:

[0028] Collect the accuracy of the second largest language model and natural language inference model in judging triples in different fields and set weight benchmarks;

[0029] Among them, the weight benchmark is dynamically adjusted according to the domain type characteristics and text length characteristics of the input text;

[0030] According to the weight reference, combining the first hallucination probability and the second hallucination probability into a final hallucination probability;

[0031] A threshold is set. If the final hallucination probability is greater than the threshold, the triple is judged to be hallucinatory.

[0032] Furthermore, outputting the final hallucination probability of each triplet includes: generating a detection report and visualizing it.

[0033] The present invention also provides a large language model hallucination detection system based on a knowledge graph structure, comprising:

[0034] Knowledge graph structure extraction module: used to obtain text data generated by a large language model, extract triples of the data, and obtain a knowledge graph triple representation set;

[0035] Hallucination detection module: Based on the large language model and natural language inference model, it determines whether there are hallucinations in the triples in the knowledge graph triple representation set;

[0036] Result visualization module: used to output the probability of hallucination for each triple and display it visually.

[0037] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the method for detecting hallucinations of a large language model based on a knowledge graph structure is implemented.

[0038] The beneficial effects of the present invention are:

[0039] 1. By converting unstructured text into knowledge graph triple representation, structured semantic analysis of LLM output content is achieved, overcoming the limitations of traditional text similarity matching or keyword retrieval, and enabling in-depth understanding of the semantic consistency of the text.

[0040] 2. Combining the semantic reasoning ability of LLM and the logical relationship judgment of NLI model, through the weighted fusion algorithm, the weight parameters are adaptively and dynamically adjusted according to the text domain type and length characteristics to integrate the advantages of the two types of models, so as to optimize the detection performance in different scenarios and significantly improve the accuracy and reliability of hallucination detection.

[0041] 3. Accurately locate the hallucination content and make the judgment basis transparent. Through knowledge graph visualization and text highlighting, the hallucination location is intuitively displayed. The verification score and contextual basis of each triple are listed to enhance the credibility and interpretability of the results.

[0042] 4. It supports text detection in different fields such as science and technology, medical care, and law, and dynamically optimizes verification strategies through domain classifiers, making it widely applicable and extensible. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0044] Figure 1 It is a schematic diagram of the overall technical route of the present invention;

[0045] Figure 2 It is a schematic flow chart of the main steps of the method of the present invention;

[0046] Figure 3 This is a schematic diagram of the dynamic weight calculation process of the present invention;

[0047] Figure 4 This is a schematic diagram of weighted calculation for hallucination determination according to the present invention;

[0048] Figure 5 It is a schematic diagram of the overall framework of the system of the present invention. DETAILED DESCRIPTION

[0049] The terms used in the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "the" and "the" used in the present invention and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0050] It should be understood that although the terms first, second, third, etc. may be used in the present invention to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0051] The present invention is described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the features of the following embodiments and implementations can be combined with each other.

[0052] like Figure 5 As shown, the embodiment of the present invention provides a large language model hallucination detection system based on a knowledge graph structure, including:

[0053] Knowledge graph structure extraction module: used to obtain text data generated by a large language model, extract triples of the data, and obtain a knowledge graph triple representation set.

[0054] Hallucination detection module: Based on the large language model and natural language inference model, it determines whether there are hallucinations in the triples in the knowledge graph triple representation set.

[0055] Result visualization module: used to output the probability of hallucination for each triple and display it visually.

[0056] like Figure 1 and Figure 2 As shown, the embodiment of the present invention provides a large language model hallucination detection method based on a knowledge graph structure, which is implemented based on the above-mentioned hallucination detection system and includes the following steps:

[0057] Step 1: Knowledge Graph Representation

[0058] Convert the unstructured text output by LLM into a structured knowledge graph triple representation. The specific steps include:

[0059] 1.1 Triplet Extraction

[0060] Prompt guides another LLM to extract entities and their relationships from unstructured natural language text and generate triples (Subject, Predicate, Object) in RDF format. The process mainly includes three steps: extract entities and their relationships. Set constraints and output triples that conform to RDF format. Format the output to ensure consistent structure.

[0061] When extracting entities and their relationships, in order to improve the extraction effect, examples are provided in the prompt to help the model understand the task. For example:

[0062] Please extract RDF triples (Subject, Predicate, Object) from the following text.

[0063] Example:

[0064] Text: Newton proposed the law of universal gravitation.

[0065] Output:

[0066] (Newton, proposed the law of universal gravitation)

[0067] Text: Einstein proposed the theory of special relativity in 1905.

[0068] Output:

[0069] (Einstein, proposed, special theory of relativity)

[0070] (Special Theory of Relativity, proposed in 1905)

[0071] Now, extract the RDF triples of the following text:

[0072] {Enter text}

[0073] Please give the extraction results:

[0074] Reduce model generation errors by clarifying constraints. Examples of constraints are as follows:

[0075] Do not leave out any important entities or relationships.

[0076] Make sure all outputs are triplets.

[0077] If there is no clear relationship in the text, omit that section rather than guessing.

[0078] Finally, specify the output format to avoid generating non-standardized results. At the same time, use tools or codes to verify the output format of LLM.

[0079] Please output the results strictly in the following format:

[0080] Each row represents a triple;

[0081] Use commas to separate the three parts of a triple;

[0082] No other superfluous content is included.

[0083] Post-process the generated RDF triples, such as removing redundant triples or correcting obvious errors.

[0084] If the output format of LLM does not meet the requirements, correct it through regular expressions or other methods.

[0085] To sum up, the final prompt example is:

[0086] Extract RDF triples (Subject, Predicate, Object) from the text.

[0087] Example:

[0088] Text: Newton proposed the law of universal gravitation.

[0089] Output:

[0090] (Newton, proposed the law of universal gravitation)

[0091] Text: Einstein proposed the theory of special relativity in 1905.

[0092] Output:

[0093] (Einstein, proposed, special theory of relativity)

[0094] (Special Theory of Relativity, proposed in 1905)

[0095] Now, extract the RDF triples of the following text:

[0096] "{input text}"

[0097] Please output the results strictly in the following format:

[0098] Each row represents a triple;

[0099] Use commas to separate the three parts of a triple;

[0100] No other superfluous content is included;

[0101] Start extraction.

[0102] 1.2. Triple Completion

[0103] When generating triples, in order to improve the completeness of the extracted triples and avoid the model missing some content, multiple rounds of interactive guidance are used to supplement. That is, after the first triple extraction is completed, the triple extraction results and the text to be extracted are input into the model to let LLM determine whether there are any unextracted triples. If so, please supplement them. After that, the format of the supplemented triples also needs to be verified. The process example is as follows:

[0104] User input:

[0105] Text: Turing is one of the founders of artificial intelligence and is famous for cracking the Enigma code.

[0106] Model generation:

[0107] (Turing is one of the founders of artificial intelligence)

[0108] User Addition:

[0109] Please add other unextracted triples based on the above sample format.

[0110] Model Update:

[0111] (Turing is one of the founders of artificial intelligence)

[0112] (Turing, founder, artificial intelligence)

[0113] (Turing, cracking, Enigma code)

[0114] Finally, the completed knowledge graph triple representation is obtained.

[0115] Step 2: Consistency Verification

[0116] Determine whether each triple in the knowledge graph is consistent with the given context. Specifically, the following steps are included:

[0117] 2.1 LLM Verification

[0118] The LLM verification step uses a certain LLM model to determine whether each triple in the knowledge graph is factually consistent with the given context and calculate the hallucination probability.

[0119] Design a prompt word template that contains a given context and a (head entity, relation, tail entity) triple. The given context part contains relevant information text to provide background information for judging the factual consistency of the triple. The head entity, relation, and tail entity are the three elements of the triple to be verified. By inserting these elements into the template, a complete question is formed to guide the verification LLM model to judge the factual consistency of the triple.

[0120] The inserted prompt word is input into the verification LLM model, which is another model different from the LLM model to be tested. It can be GPT-3, GPT-4 or other advanced large language models. The verification LLM model processes the prompt and uses its trained semantic understanding and reasoning capabilities to judge the factual consistency of the triple in a given context.

[0121] Formulate specific rules for verifying the output results of the LLM model and convert them into probability values ​​between [0, 1] as the hallucination probability of the triple. For example, if the output represents "correct", the result is mapped to a probability value of 0; if the output represents "wrong", the result is mapped to a probability value of 1; if the output is some vague expression, such as "may be correct" or "partially correct", the result is mapped to 0.5 or other values ​​between 0 and 1 according to the specific vocabulary.

[0122] LLM verification case. The designed prompt word template is:

[0123] Context: {given context}

[0124] Question: Based on the above context, is the statement "{head entity} {relationship} {tail entity}" correct?

[0125] Please give your judgment and specific reasons.

[0126] Given the context "Apple is a common fruit rich in vitamins" and the triple ("apple", "is", "fruit"), inserting it into the prompt word template gives:

[0127] Context: Apple is a common fruit rich in various vitamins.

[0128] Question: Based on the above context, is the statement “Apple is a fruit” correct?

[0129] Please give your judgment and specific reasons.

[0130] Verify the LLM model output: "Judgment: This statement is correct. Reason: The context clearly states that apple is a common fruit, which directly shows that apple belongs to the category of fruit." The output clearly indicates "correct", so the result is mapped to a probability value of 0.

[0131] 2.2 NLI Model Validation

[0132] The NLI model verification step uses an NLI model to determine whether each triple in the knowledge graph is factually consistent with the given context and calculates the hallucination probability.

[0133] Convert the (head entity, relation, tail entity) triple into a natural language statement as hypothesis H. Determine the given context as premise P. When the given context length plus the hypothesis H length does not exceed the maximum input length of the NLI model, the given context is directly used as premise P. When the given context length plus the hypothesis H length exceeds the maximum input length of the NLI model, extract the context information related to the triple from the given context as premise P.

[0134] The context is extracted by keyword matching and topic relevance analysis. Generate a sentence set for a given context sentence , extract the head entity and the tail entity from the triple as keywords, traverse S, for , if the keyword appears in In Add to the key sentence sequence KS, otherwise Add to the non-critical sentence sequence NS. Considering that the sum of the text lengths of the critical sentence sequence KS may exceed the maximum input length of the NLI model, it is necessary to sort the importance of the sentences in the critical sentence sequence KS. Considering that there may be important information in the non-critical sentence sequence NS that affects the judgment of the factual consistency of the triples, it is necessary to sort the importance of the sentences in the non-critical sentence sequence NS. The importance is sorted by calculating the relevance between the sentence and the triple, and sorting from large to small. The calculation of the relevance between the sentence and the triple is completed by training a deep learning neural network and using cosine similarity as the measurement method. Traverse KS and , calculate the sentence Similarity to triples , generate key sentence similarity sequence The final set of sentences selected is R, which is initially an empty set. For KDS, take the index j of the largest value and replace the sentence with index j in KS As a candidate sentence, if the sum of the text length of the candidate sentence and all sentences in R does not exceed the maximum input length of the NLI model, the candidate sentence is added to R; continue to take the index k of the second largest value and add the sentence with index k in KS As a candidate sentence, and so on, until the text length of a candidate sentence and all sentences in R exceeds the maximum input length of the NLI model or the KDS is traversed. If the text length of all sentences in R does not exceed the maximum input length of the NLI model, traverse NS and , calculate the sentence Similarity to triples , generate non-key sentence similarity sequence For NDS, take the index j of the largest value, if If it is greater than the threshold, the index j in NS As a candidate sentence, if the sum of the text length of the candidate sentence and all sentences in R does not exceed the maximum input length of the NLI model, the candidate sentence is added to R; continue to take the index k of the second largest value, if If it is greater than the threshold, the index k in NS As a candidate sentence, and so on, until d is less than the threshold or the text length of a candidate sentence and all sentences in R exceeds the maximum input length of the NLI model or the NDS is traversed. The sentences in R are concatenated into text in the order in the given context as the premise P.

[0135] The hypothesis H and premise P are input into the selected NLI model. The NLI model uses a pre-trained natural language inference model such as BERT. The model uses a neural network to infer the logical relationship between the hypothesis and the premise, and outputs the probability distribution of the three relationships of implication, contradiction, and neutrality. The contradiction probability output by the model is used as the hallucination probability of the triple.

[0136] 2.3 Comprehensive judgment

[0137] like Figure 3 and Figure 4 As shown, the comprehensive judgment step is to combine the hallucination probability output by the LLM model and the hallucination probability output by the NLI model to finally determine whether the triple is a hallucination.

[0138] The combined formula is:

[0139] final_score = α*llm_score+(1-α)*nli_score

[0140] Among them, α is an adjustable weight parameter, llm_score is the hallucination probability output by the LLM model, and nli_score is the hallucination probability output by the NLI model.

[0141] First, determine the weight benchmark α based on iteratively updated historical verification data. Collect enough data, which should cover different fields, different natural language tasks, and different text lengths. The fields include science and technology, medicine, law, finance, literature, etc. Natural language tasks include text classification tasks, information extraction tasks, etc. Different text lengths cover short texts, medium texts, and long texts to ensure the comprehensiveness and reliability of the data. Use the LLM model to process the collected data to obtain the prediction results, and calculate the accuracy accuracyLLM based on the prediction results and annotations.

[0142] .

[0143] Similarly, for the NLI model, calculate the accuracy accuracyNLI.

[0144] .

[0145] Taking the above two accuracy rates as the initial weight benchmark, the initial value of α is preliminarily set as:

[0146] .

[0147] Secondly, α is dynamically adjusted according to the input text features. The input text features include domain type features and text length features.

[0148] The input text is classified into different fields, such as science and technology, medical treatment, and law, using pre-trained field classifiers. Different weight adjustment coefficients are set for different field types. Through the analysis of historical verification data, the performance differences between LLM and NLI models in different fields are found. The adjustment formula of α is:

[0149] .

[0150] Among them, i is the field, is the weight parameter of domain i, is the adjustment coefficient of domain i, reflecting the degree of deviation of the performance difference between LLM and NLI in domain i relative to the global one. Its value is calculated based on the historical verification data of domain i:

[0151] .

[0152] Among them, the molecule represents the absolute performance difference between LLM and NLI in domain i; the denominator Incorporate normalized differences to avoid deviations caused by the absolute value of the accuracy. Then, based on the global initial weight Adjust the coefficient range to ensure that the adjustment range is consistent in all areas.

[0153] The text length of the input text is calculated and divided into short text, medium text and long text. Different weight adjustment coefficients are set for texts of different length types. Through the analysis of historical verification data, the performance differences between LLM and NLI models in different length types are found. The adjustment formula of α is:

[0154] .

[0155] Among them, j represents three types of text lengths, is the adjustment coefficient for the corresponding text length type. Its value is calculated based on the historical verification data of the corresponding text length type. The calculation process is the same as the field adjustment coefficient. Similarly, The weight parameter corresponding to the text length type.

[0156] For input text belonging to domain i and text length type j, its weight α is:

[0157]

[0158] Among them, i is the field, is the adjustment coefficient of domain i, and its value is calculated based on the historical verification data of domain i. j represents three types of text lengths. is the adjustment coefficient for the corresponding text length type, and its value is calculated based on the historical verification data for the corresponding text length type. is the weight parameter of text length type j in domain i.

[0159] Set a threshold. If the final_score is greater than the threshold, the triple is considered to be a hallucination. If any triple in the knowledge graph is considered to be a hallucination, the overall output is marked as inconsistent.

[0160] 2.4 Result Output

[0161] The result output step includes generating a test report and visual display.

[0162] After hallucination detection on the large model output, the detection results need to be presented to the user. In order to facilitate user understanding and analysis, the result output step includes generating a detection report and visual display.

[0163] The inspection report provides a comprehensive assessment of the consistency of the large model output and lists in detail all hallucinations detected. The report content structure includes:

[0164] Overall consistency judgment result. The final judgment result is clearly given as "consistent" or "inconsistent (hallucinations exist)". If hallucinations exist, the severity of the hallucinations is further described, and based on the number and proportion of inconsistent triplets, it is described as "mild hallucinations", "moderate hallucinations" or "severe hallucinations".

[0165] like Figure 4 As shown, each triplet and its detailed score are recorded. All triples extracted from the LLM output are listed, and the hallucination probability llm_score based on the LLM model, the hallucination probability nli_score based on the NLI model, and the weighted final score final_score of each triplet are given.

[0166] List of triplets with hallucinations and their positions. List all triplets that are judged to be hallucinations. For easy positioning, mark the position of each inconsistent triple in the original text with the sentence number.

[0167] Visual display presents the detection results in a more intuitive way, mainly including text highlighting and knowledge graph visualization.

[0168] Text highlighting. In the original text, the text paragraphs with hallucinations are highlighted, and different colors are used to mark different degrees of hallucinations. For example, yellow marks mild hallucinations, orange marks moderate hallucinations, and red marks severe hallucinations. When the mouse hovers over the highlighted text, the corresponding triples and their detailed scores are displayed.

[0169] Knowledge graph visualization. The knowledge graph extracted from the LLM output is presented graphically, with the entities of inconsistent triples highlighted in red and the relations in bold. When the mouse hovers over an inconsistent triple, the detailed score of the triple is displayed.

[0170] An embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the method for detecting hallucinations of a large language model based on a knowledge graph structure is implemented.

[0171] The computer-readable storage medium may be an internal storage unit of any device with data processing capability described in any of the aforementioned embodiments, such as a hard disk or a memory. The computer-readable storage medium may also be any device with data processing capability, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of any device with data processing capability and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capability, and may also be used to temporarily store data that has been output or is to be output.

[0172] Those skilled in the art will readily appreciate other embodiments of the present application after considering the description and practicing the contents disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application. The description and examples are intended to be exemplary only.

[0173] It should be understood that the present application is not limited to the exact construction that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof.

Claims

1. A large language model hallucination detection method based on knowledge graph structure, characterized in that: include: Acquire text data output by the first language model, and generate triples in RDF format according to the text data; Through multiple rounds of interactions, the triples of text data are supplemented to obtain a knowledge graph triple representation set; Based on the second language model, the consistency between each triple in the knowledge graph triple representation set and the corresponding text context is judged, and the first hallucination probability is calculated; Based on the natural language inference model, the consistency between each triple in the knowledge graph triple representation set and the corresponding text context is determined, and the second hallucination probability is calculated; According to the first hallucination probability and the second hallucination probability, a final hallucination probability is obtained to determine whether the hallucination of the same triple is true; Output the final hallucination probability of each triplet.

2. According to the method for detecting hallucinations in a large language model based on a knowledge graph structure according to claim 1, it is characterized in that: The step of obtaining text data output by the first language model and generating triples in RDF format according to the text data includes: The first language model is guided by prompt to extract entities and their relationships from unstructured natural language text, and constraints are set to generate triples that conform to the RDF format; The generated triples are verified and post-processed, and the erroneous triples are corrected.

3. According to claim 2, a large language model hallucination detection method based on a knowledge graph structure is characterized in that: The extraction of entities and their relations includes providing examples in prompts to help the model understand the task.

4. According to the method for detecting hallucinations in a large language model based on a knowledge graph structure according to claim 1, it is characterized in that: The triples of text data are supplemented through multiple rounds of interaction to obtain a knowledge graph triple representation set including: After the triples are generated for the first time, the triples and the corresponding texts are input into the first largest language model to determine whether there are any unextracted triples. If so, they are supplemented and the supplemented triples are verified and corrected to obtain a set of triple representations of the completed knowledge graph.

5. According to the method for detecting hallucinations in a large language model based on a knowledge graph structure as described in claim 1, it is characterized in that: The step of judging the consistency between each triple in the knowledge graph triple representation set and the corresponding text context based on the second largest language model and calculating the first hallucination probability includes: Setting a prompt word template, including a triple and a corresponding text context, wherein the text context includes a head entity, a relation, and a tail entity; The prompt word is input into the second largest language model, the consistency of the triple is judged by the text context, and the output result is mapped to the first hallucination probability; if correct, the result is mapped to a probability value of 0, if wrong, the result is mapped to a probability value of 1, and if partially correct, the result is mapped to a probability value between 0 and 1 according to the part corresponding to the triplet and the text context.

6. According to the method for detecting hallucinations in a large language model based on a knowledge graph structure according to claim 1, it is characterized in that: The method of judging the consistency between each triple in the knowledge graph triple representation set and the corresponding text context based on the natural language inference model and calculating the second hallucination probability includes: Convert the triple into a natural language statement as a hypothesis H. When the sum of the length of the text context corresponding to the triple and the length of the hypothesis H does not exceed the maximum input length of the natural language inference model, use the text context directly as the premise P. When the sum of the length of the text context and the length of the hypothesis H exceeds the maximum input length of the natural language inference model, extract context information related to the triple from the text context as the premise P. The hypothesis H and premise P are input into a natural language reasoning model, and the logical relationship between the hypothesis H and the premise P is inferred through the natural language reasoning model, and the probability distribution of three relationships: implication, contradiction and neutrality is output; the contradiction probability output by the model is used as the second hallucination probability of the triple.

7. According to the method for detecting hallucinations in a large language model based on a knowledge graph structure as claimed in claim 1, it is characterized in that: The step of obtaining a final hallucination probability according to the first hallucination probability and the second hallucination probability, and judging whether the hallucination of the same triplet is true comprises: Collect the accuracy of the second largest language model and natural language inference model in judging triples in different fields and set weight benchmarks; Among them, the weight benchmark is dynamically adjusted according to the domain type characteristics and text length characteristics of the input text; According to the weight reference, combining the first hallucination probability and the second hallucination probability into a final hallucination probability; A threshold is set. If the final hallucination probability is greater than the threshold, the triple is judged to be hallucinatory.

8. According to the method for detecting hallucinations in a large language model based on a knowledge graph structure as claimed in claim 1, it is characterized in that: The outputting of the final hallucination probability of each triplet includes: generating a detection report and visualizing it.

9. A large language model hallucination detection system based on knowledge graph structure, characterized in that: include: Knowledge graph structure extraction module: used to obtain text data generated by a large language model, extract triples of the data, and obtain a knowledge graph triple representation set; Hallucination detection module: Based on the large language model and natural language inference model, it determines whether there are hallucinations in the triples in the knowledge graph triple representation set; Result visualization module: used to output the probability of hallucination for each triple and display it visually.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, a large language model hallucination detection method based on a knowledge graph structure is implemented as described in any one of claims 1 to 8.

Images