Hallucination detection method and device based on semantic embedding and structural entropy detection

Abstract

This application proposes a hallucination detection method and device based on semantic embedding and structural entropy detection; the method includes: for an input question, using a trained language model to generate multiple possible answer sets; using a pre-trained sentence embedding model to convert each answer into a vector, calculating the cosine similarity between them, and thus constructing a similarity graph; representing the similarity graph through a coding tree, calculating the structural entropy of each node to identify nodes with high semantic uncertainty; using a sliding average method to dynamically adjust the threshold to ensure that high entropy outputs are marked as hallucinations in a timely manner; and eliminating answers marked as hallucinations to ensure the accuracy of the output answers and the consistency of the information. The present invention effectively improves the performance of the model on complex problems and reduces the generation of misleading information. Preliminary experiments show that this method significantly reduces the proportion of hallucination outputs on multiple data sets and enhances the semantic consistency and accuracy of the generated answers.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing, and in particular to a hallucination detection method and device based on semantic embedding and structural entropy detection. Background Art

[0002] In the medical field, especially in scenarios involving disease diagnosis and treatment consultation, the accuracy of information is crucial. For example, in the diagnosis and treatment recommendations for esophageal cancer, incorrect information may not only mislead patients but also endanger their lives. Therefore, ensuring the reliability and accuracy of content generated by large language models has become an important topic of current research. With the rapid development of artificial intelligence technology, significant progress has been made in the field of natural language processing (NLP). Large language models (LLMs) have been widely used in various tasks such as question-answering systems, generative dialogue, text summarization, and machine translation due to their powerful text generation and understanding capabilities. However, these models still face the risk of false or inaccurate output when generating content, especially when involving sensitive areas such as medicine, law, and finance, where this risk can lead to serious consequences.

[0003] Existing methods still have certain limitations in practical applications, making it difficult to fully address the problem of false information detection. Therefore, it is particularly important to develop a new hallucination detection system that integrates semantic embedding and structural entropy. This innovation can not only improve the performance of large-scale language models in medical question-answering, but also reduce the risk of false output in high-risk fields such as healthcare, thereby providing users with safer and more reliable services. To address this problem, researchers have begun exploring various technical approaches. Structural entropy, a key concept in information theory, effectively measures the distribution and uncertainty of information. By analyzing the information structure in text, it helps identify potential false output. Simultaneously, semantic embedding technology further enhances the credibility of content by analyzing the semantic similarity between generated content and real data. The combination of these technologies provides new insights for improving the reliability of large-scale language models. Summary of the Invention

[0004] To solve the above problems, the present invention provides a hallucination detection method based on semantic embedding and structural entropy detection.

[0005] To implement the above technical solution, the specific steps are as follows:

[0006] A hallucination detection method based on the fusion of semantic embedding and structural entropy includes the following steps:

[0007] S1. Collect data related to esophageal cancer. Through a multi-dimensional data collection strategy, a rich and reliable data foundation is provided for subsequent hallucination detection, ensuring that the system's evaluation in esophageal cancer detection is more scientific and effective.

[0008] To effectively evaluate the performance of large-scale language models in esophageal cancer detection, data collection is crucial. This process aims to obtain relevant questions and answers to analyze the accuracy and reliability of the content generated by the model.

[0009] The collected data include: clinical data, database data and real case data;

[0010] Clinical data includes: clinical questions and standard answers; clinical questions are collected from actual problems encountered by patients and doctors, such as "What are the early symptoms of esophageal cancer?" and "What are the common treatments?"; standard answers are collected from authoritative medical literature and clinical practice.

[0011] The database includes medical literature, clinical guidelines, and an online Q&A platform. The medical literature refers to research on esophageal cancer, obtained through the review of relevant academic papers and systematic reviews, and provides corresponding solutions. The clinical guidelines extract information on esophageal cancer research from clinical guidelines published by the American Cancer Society. The online Q&A platform uses HealthTap and WebMD to collect interactive Q&A between esophageal cancer patients and doctors, reflecting real clinical questions.

[0012] Real case data refers to the collection of symptom descriptions and diagnosis results of patients diagnosed with esophageal cancer, including: medical records of medical institutions and case reports in professional journals;

[0013] These data provide the symptoms and diversity of real cases and help frame the relevant questions;

[0014] S2. Preprocess the collected data;

[0015] Data preprocessing is a critical step to ensure that the answers generated by the model are consistent and accurate with the clinical questions; it includes the following steps:

[0016] S2.1. Data cleaning: The goal of the data cleaning stage is to remove noise and inconsistencies in the collected data to ensure data purity and usability. The specific steps are as follows:

[0017] S2.1.1. Remove redundant characters and potential interference information to make the data easier to analyze;

[0018] Specifically, removing redundant characters means cleaning up special characters and irrelevant punctuation marks in text data to ensure the neatness of text content;

[0019] S2.1.2. Correct spelling errors; accurate text input can improve the learning effect of subsequent models;

[0020] Specifically, use spell-checking tools to identify and correct spelling errors in text to enhance the readability and accuracy of data;

[0021] S2.1.3. Handling missing values ​​ensures data integrity and avoids analytical bias caused by missing information;

[0022] Specifically, by manually identifying missing clinical questions or answers in the collected data, we decide whether to fill in, delete, or keep them as they are to ensure the integrity of the dataset;

[0023] S2.2. Data standardization: Convert the cleaned data into a unified format to facilitate subsequent analysis and input. The specific steps are as follows:

[0024] S2.2.1. Text normalization makes the model more efficient when processing text;

[0025] Specifically, all text data is converted to lowercase to reduce the impact of case differences on analysis, and redundant spaces and line breaks are removed to ensure consistency in text format.

[0026] S2.2.2. Numerical standardization: By converting numerical data to the same scale, the model training effect is improved and each feature is equally important in the model learning;

[0027] Specifically, numerical data related to patient symptoms (such as age and tumor size) were standardized to eliminate the impact of different dimensions on the analysis;

[0028] Through data preprocessing, a solid data foundation was established to support the accuracy and reliability of the model in esophageal cancer detection. The processed data was used to determine hyperparameters.

[0029] S3. Perform hallucination detection on the processed data and output the final answer; the steps are as follows:

[0030] S3.1. Input the processed data into a large language model (GPT-4) to train the model;

[0031] S3.2. Build a similarity graph between the answers generated by the trained model. The generated similarity graph is used for the subsequent calculation of structural entropy. The steps are as follows:

[0032] S3.2.1. Input the question into the trained model and generate multiple corresponding answers s (i) and collect answers (i) The answer set A is formed, and the expression is as follows:

[0033] A=s (i) , i depends on the number of problems, where i∈(1,N), N is a natural integer;

[0034] S3.2.2, using semantic embedding detection method to generate semantic similarity between answers;

[0035] Specifically, the sentence vector extraction method based on the BERT model (Sentence-BERT) is used to extract each generated answer s (i) Convert to embedding vector e (i) , and use cosine similarity to calculate semantic similarity;

[0036] The conversion expression is as follows:

[0037] e (i) =SBERT(s (i) );

[0038] The expression of cosine similarity is as follows:

[0039] i depends on the number of questions;

[0040] i, j∈(1, N), N is a natural integer, i≠j

[0041] Where, e (i) ·e (j) Represents the dot product of vectors, which can reflect the similar directions of two vectors; ‖e (i) ‖‖e (j) ‖ is the norm of the vector, which ensures that the similarity value is between 0 and 1. The closer the value is to 1, the more similar it is.

[0042] S3.2.3. Construct a similarity graph using the cosine similarity results.

[0043] Specifically, a similarity graph is obtained by adding a weighted edge to the undirected graph G;

[0044] The expression of the undirected graph G is as follows:

[0045] G=(V,E)

[0046] Where V is the vertex set, that is, each generated answer s (i) Corresponding to a vertex in the graph; E is the edge set, which is calculated by calculating the cosine similarity between the answers and adding edges based on the cosine similarity. The edge weights between answers with high cosine similarity are higher, and vice versa.

[0047] Furthermore, the threshold θ is set based on empirical statistics. b , used to determine whether two answers are similar enough to establish a connection; for each pair of answers s (i) and s (j) , if its similarity is higher than the threshold θ b , then add a weighted edge to the undirected graph G, where the weight is the cosine similarity value; the expression is as follows:

[0048] if sim(s (i) , s (j) )>θ b ;

[0049] G.add_edge(s (i) , s (j) ,weight=sim(s (i) , s (j) ))

[0050] In the formula, weight is the weight;

[0051] S3.3. Calculate the structural entropy based on the similarity graph; the expression is as follows:

[0052]

[0053] Where H T (G; α) is the structural entropy; g α is the sum of the weights of the edges connected to node α (the generated answer or the result of a semantic clustering); m is the sum of the weights of all edges in the undirected graph; V α is the number of vertices in node α; V α - is the number of vertices in the parent node of node α;

[0054] Furthermore, the coding tree T is a tree structure representation of the undirected graph G. Each node of the coding tree corresponds to a non-empty vertex subset in V. The root node represents the entire vertex set of the undirected graph G. Each leaf node corresponds to a vertex in the undirected graph G, i.e., a generated answer.

[0055] The total structural entropy of the undirected graph G is the sum of the entropies of all nodes in the coding tree T; the expression is as follows:

[0056] H T (G) = ∑ α∈T H T (G;α);

[0057] By minimizing structural entropy, the system is able to identify semantically equivalent groups of generated answers;

[0058] S3.4, Detecting hallucinations by optimizing structural entropy;

[0059] Specifically, by presetting a threshold, all nodes or subgraphs whose structural entropy exceeds the threshold are marked as hallucination outputs, and these nodes and their corresponding generated answers are recorded to form the hallucination output set S H The specific steps are as follows:

[0060] S3.4.1, Optimize structural entropy;

[0061] After calculating the structural entropy of each node or subgraph, the next step is to set a reasonable threshold. The choice of this threshold is crucial, as it not only affects the detection of hallucinated outputs but also directly affects the overall performance of the system. When setting the threshold, three factors need to be considered: empirical basis, data characteristics, and dynamic adjustment.

[0062] The empirical basis is to select a threshold that can effectively distinguish normal and abnormal outputs based on past experience and experimental evidence. It is necessary to perform statistics on the structural entropy distribution on multiple data sets and determine its average value. and standard deviation σ H ; Initial threshold θ u Can be set to:

[0063]

[0064] Wherein, k is an adjustment parameter, which usually ranges from 1.0 to 2.0. In this embodiment, in order to be as comprehensive as possible and avoid mislabeling, it is set to 1.5;

[0065] Then, by selecting a threshold suitable for the specific task and data, we analyze the data characteristics and the diversity and complexity of the generated answers. Different types of questions may lead to different structural entropy distributions, so the setting of the initial threshold should be flexible. The initial threshold is further designed and expressed as follows:

[0066]

[0067] Where, is the average structural entropy of the number of problems i;

[0068] Dynamic adjustment means that in actual applications, the threshold is flexibly and dynamically adjusted based on real-time feedback and system performance. For example, if the system finds that certain high-entropy outputs are often marked as hallucination outputs and affect the user experience, it is necessary to use the sliding average method to mark the initial threshold with flexibility. The initial threshold with flexibility is further designed, and the expression is as follows:

[0069] θ t+1 =(1-a)·θ t +a·H t

[0070] Where a is the smoothing parameter (the value of the smoothing parameter should be in the range of 0.1 to 0.7, determined by verifying its performance), H t The average structural entropy calculated for the latest round

[0071] The determined threshold is used as a criterion for determining whether the answer output by the model is hallucination or not;

[0072] By setting the threshold appropriately, we can effectively identify nodes or subgraphs with high semantic uncertainty, thus laying the foundation for subsequent labeling and processing.

[0073] S3.4.2. Output hallucination marking, including: identifying high entropy nodes, marking and recording, and rejecting output;

[0074] To identify high entropy nodes, the system automatically traverses each node and checks whether its structural entropy is higher than the set threshold;

[0075] Marking and recording: Mark the nodes that exceed the threshold and record relevant information, including: node characteristics, generated answer content, and structural entropy value;

[0076] Reject output means that for nodes marked as hallucination output, the system should reject the corresponding answer; this mechanism can effectively reduce the misleading information received by users, thereby improving the accuracy of the overall output;

[0077] Hallucination Mark Results S H , contains all answers marked as hallucinated output;

[0078] S3.5. Output the correct answer based on the hallucination markup. The steps are as follows:

[0079] S3.5.1, Eliminate hallucination output; According to the hallucination tag set S H , remove all answers marked as hallucination output from the answer set A to form a clean answer set A clean , the expression is as follows:

[0080] A clean =A / S H ;

[0081] S3.5.2, answer regeneration;

[0082] Specifically, for the input problem marked as hallucination, if A clean The answers in the query are insufficient to answer the original question, or too many answers are removed, resulting in incomplete information. The model needs to regenerate new answers.

[0083] The regeneration step includes: re-inputting the original question into the model and outputting the labeled hallucination S H Provided as negative samples to the model to help the model avoid generating similar hallucinated answers and for the regenerated answers, repeat embedding similarity detection and structural entropy calculation to ensure that the newly generated answers meet the set entropy threshold requirements;

[0084] S3.5.3. Output the final answer;

[0085] Comprehensive A cleanand the new answers obtained through the regeneration process to form the final answer set A final , ensuring that all answers are not marked as hallucination outputs and meet semantic consistency and accuracy, and finally output the correct answer set A final .

[0086] Beneficial effects of the present invention

[0087] 1. This paper significantly improves the reliability and accuracy of the medical question-answering system related to esophageal cancer by integrating semantic embedding and structural entropy detection.

[0088] 2. The present invention intuitively displays the relationships and similarities between different treatment options by integrating graph structure and entropy calculation, providing more personalized and accurate medical services. At the same time, it effectively detects and reduces the risk of the model generating hallucination information, thereby improving the accuracy of diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] Figure 1 is a flow chart of the present invention;

[0090] Figure 2 It is the overall framework diagram of the present invention;

[0091] Figure 3 Data collection and preprocessing for the present invention;

[0092] Figure 4 is a flow chart of hallucination detection of the present invention;

[0093] Figure 5 This is a flowchart of the answer to the discarded illusion of the present invention. DETAILED DESCRIPTION

[0094] The present invention is further described in detail below with reference to specific embodiments.

[0095] like Figure 1-2 As shown, a hallucination detection method based on the fusion of semantic embedding and structural entropy includes the following steps:

[0096] S1. Collect data related to esophageal cancer. Through a multi-dimensional data collection strategy, a rich and reliable data foundation is provided for subsequent hallucination detection, ensuring that the system's evaluation in esophageal cancer detection is more scientific and effective.

[0097] To effectively evaluate the performance of large-scale language models in esophageal cancer detection, data collection is crucial. This process aims to obtain relevant questions and answers to analyze the accuracy and reliability of the content generated by the model.

[0098] The collected data include: clinical data, database data and real case data;

[0099] Clinical data includes: clinical questions and standard answers; clinical questions are collected from actual problems encountered by patients and doctors, such as "What are the early symptoms of esophageal cancer?" and "What are the common treatments?"; standard answers are collected from authoritative medical literature and clinical practice.

[0100] The database includes medical literature, clinical guidelines, and an online Q&A platform. The medical literature refers to research on esophageal cancer, obtained through the review of relevant academic papers and systematic reviews, and provides corresponding solutions. The clinical guidelines extract information on esophageal cancer research from clinical guidelines published by the American Cancer Society. The online Q&A platform uses HealthTap and WebMD to collect interactive Q&A between esophageal cancer patients and doctors, reflecting real clinical questions.

[0101] Real case data refers to the collection of symptom descriptions and diagnosis results of patients diagnosed with esophageal cancer, including: medical records of medical institutions and case reports in professional journals;

[0102] These data provide the symptoms and diversity of real cases and help frame the relevant questions;

[0103] S2, such as Figure 3 As shown in Figure 1, the collected data is preprocessed. Data preprocessing is a key step to ensure that the answers generated by the model are consistent and accurate with the clinical questions. It includes the following steps:

[0104] S2.1. Data cleaning: The goal of the data cleaning stage is to remove noise and inconsistencies in the collected data to ensure data purity and usability. The specific steps are as follows:

[0105] S2.1.1. Remove redundant characters and potential interference information to make the data easier to analyze;

[0106] Specifically, removing redundant characters means cleaning up special characters and irrelevant punctuation marks in text data to ensure the neatness of text content;

[0107] S2.1.2. Correct spelling errors; accurate text input can improve the learning effect of subsequent models;

[0108] Specifically, use spell-checking tools to identify and correct spelling errors in text to enhance the readability and accuracy of data;

[0109] S2.1.3. Handling missing values ​​ensures data integrity and avoids analytical bias caused by missing information;

[0110] Specifically, by manually identifying missing clinical questions or answers in the collected data, we decide whether to fill in, delete, or keep them as they are to ensure the integrity of the dataset;

[0111] S2.2. Data standardization: Convert the cleaned data into a unified format to facilitate subsequent analysis and input. The specific steps are as follows:

[0112] S2.2.1. Text normalization makes the model more efficient when processing text;

[0113] Specifically, all text data is converted to lowercase to reduce the impact of case differences on analysis, and redundant spaces and line breaks are removed to ensure consistency in text format.

[0114] S2.2.2. Numerical standardization: By converting numerical data to the same scale, the model training effect is improved and each feature is equally important in the model learning;

[0115] Specifically, numerical data related to patient symptoms (such as age and tumor size) were standardized to eliminate the impact of different dimensions on the analysis;

[0116] Through data preprocessing, a solid data foundation was established to support the accuracy and reliability of the model in esophageal cancer detection. The processed data was used to determine hyperparameters.

[0117] S3, such as Figure 4 As shown, hallucination detection is performed on the processed data and the final answer is output; the steps are as follows:

[0118] S3.1. Input the processed data into a large language model (GPT-4) to train the model;

[0119] S3.2. Build a similarity graph between the answers generated by the trained model. The generated similarity graph is used for the subsequent calculation of structural entropy. The steps are as follows:

[0120] S3.2.1. Input the question into the trained model and generate multiple corresponding answers s (i) and collect answers (i) The answer set A is formed, and the expression is as follows:

[0121] A=s (i) , i depends on the number of problems, where i∈(1,N), N is a natural integer;

[0122] These generated answers are used for semantic clustering and structural entropy calculation, by comparing and clustering the semantic similarities of the answers in order to discover potential themes or meaning clusters;

[0123] In addition, structural entropy calculation is used to evaluate the diversity and information content of the generated answers. By calculating the structural entropy of the answers, researchers can understand the complexity and information richness of the answers, and further analyze the diversity and quality of the answers generated by the model.

[0124] The comprehensive use of the probability distribution and semantic clustering of these generated answers can help us gain a deeper understanding of the model's response to specific input questions and the characteristics of the generated answers, providing more useful information for subsequent applications and analysis.

[0125] S3.2.2, using semantic embedding detection method to generate semantic similarity between answers;

[0126] Semantic embedding detection methods can effectively capture the semantic similarity between generated answers, helping users to deeply analyze and understand the content generated by the model. This method not only improves the clustering effect of answers, but also provides rich semantic information for subsequent applications.

[0127] Specifically, the sentence vector extraction method based on the BERT model (Sentence-BERT) is used to extract each generated answer s (i) Convert to embedding vector e (i) , and use cosine similarity to calculate semantic similarity;

[0128] The conversion expression is as follows:

[0129] e (i) =SBERT(s (i) );

[0130] The expression of cosine similarity is as follows:

[0131] i depends on the number of questions;

[0132] i, j∈(1, N), N is a natural integer, i≠j

[0133] Where, e (i) ·e (j) Represents the dot product of vectors, which can reflect the similar directions of two vectors; ||e (i) ||||e (j) || is the norm of the vector, which ensures that the similarity value is between 0 and 1. The closer the value is to 1, the more similar it is.

[0134] S3.2.3. Construct a similarity graph using the cosine similarity results.

[0135] Specifically, a similarity graph is obtained by adding a weighted edge to the undirected graph G;

[0136] The expression of the undirected graph G is as follows:

[0137] G=(V,E)

[0138] Where V is the vertex set, that is, each generated answer s (i)Corresponding to a vertex in the graph; E is the edge set, which is calculated by calculating the cosine similarity between the answers and adding edges based on the cosine similarity. The edge weights between answers with high cosine similarity are higher, and vice versa.

[0139] Furthermore, the threshold θ is set based on empirical statistics. b , used to determine whether two answers are similar enough to establish a connection; for each pair of answers s (i) and s (j) , if its similarity is higher than the threshold θ b , then add a weighted edge to the undirected graph G, where the weight is the cosine similarity value; the expression is as follows:

[0140] if sim(s (i) , s (j) )>θ b ;

[0141] G.add_edge(s (i) , s (j) ,weight=sim(s (i) , s (j) ))

[0142] In the formula, weight is the weight;

[0143] S3.3. Calculate the structural entropy based on the similarity graph. Structural entropy is an indicator that measures the complexity and uncertainty of information and can reveal the semantic diversity of the generated answers. The expression is as follows:

[0144]

[0145] Where H T (G; α) is the structural entropy; g α is the sum of the weights of the edges connected to node α (the generated answer or the result of a semantic clustering); m is the sum of the weights of all edges in the undirected graph; V α is the number of vertices in node α; V α - is the number of vertices in the parent node of node α;

[0146] Furthermore, the coding tree T is a tree structure representation of the undirected graph G. Each node of the coding tree corresponds to a non-empty vertex subset in V. The root node represents the entire vertex set of the undirected graph G. Each leaf node corresponds to a vertex in the undirected graph G, i.e., a generated answer.

[0147] In this embodiment, the root node represents the set of all possible answers related to esophageal cancer; the leaf nodes represent the specific answers actually generated; the vertices represent specific topics, question types, or similar answer sets;

[0148] This tree-structured representation can better organize and display the relationships and hierarchical structures between generated answers. The construction of the encoding tree can help users understand the hierarchical relationships between generated answers, gradually displaying the connections between answers from the overall to the detailed level, which helps to more comprehensively analyze and understand the structural and semantic characteristics of the generated answers.

[0149] The total structural entropy of the undirected graph G is the sum of the entropies of all nodes in the coding tree T; the expression is as follows:

[0150] H T (G) = ∑ α∈T H T (G;α);

[0151] By minimizing structural entropy, the system is able to identify semantically equivalent groups of generated answers;

[0152] S3.4, Detecting hallucinations by optimizing structural entropy;

[0153] According to the constructed similarity graph, the structural information of the similarity graph can be effectively used to calculate the structural entropy and detect hallucination output accordingly;

[0154] Specifically, by presetting a threshold, all nodes or subgraphs whose structural entropy exceeds the threshold are marked as hallucination outputs, and these nodes and their corresponding generated answers are recorded to form the hallucination output set S H The specific steps are as follows:

[0155] S3.4.1, Optimize structural entropy;

[0156] After calculating the structural entropy of each node or subgraph, the next step is to set a reasonable threshold. The choice of this threshold is crucial, as it not only affects the detection of hallucinated outputs but also directly affects the overall performance of the system. When setting the threshold, three factors need to be considered: empirical basis, data characteristics, and dynamic adjustment.

[0157] The empirical basis is to select a threshold that can effectively distinguish normal and abnormal outputs based on past experience and experimental evidence. It is necessary to perform statistics on the structural entropy distribution on multiple data sets and determine its average value. and standard deviation σ H ; Initial threshold θ u Can be set to:

[0158]

[0159] Wherein, k is an adjustment parameter, which usually ranges from 1.0 to 2.0. In this embodiment, in order to be as comprehensive as possible and avoid mislabeling, it is set to 1.5;

[0160] Then, by selecting a threshold suitable for the specific task and data, we analyze the data characteristics and the diversity and complexity of the generated answers. Different types of questions may lead to different structural entropy distributions, so the setting of the initial threshold should be flexible. The initial threshold is further designed and expressed as follows:

[0161]

[0162] Where, is the average structural entropy of the number of problems i;

[0163] Dynamic adjustment means that in actual applications, the threshold is flexibly and dynamically adjusted based on real-time feedback and system performance. For example, if the system finds that certain high-entropy outputs are often marked as hallucination outputs and affect the user experience, it is necessary to use the sliding average method to mark the initial threshold with flexibility. The initial threshold with flexibility is further designed, and the expression is as follows:

[0164] θ t+1 =(1-a)·θ t +a·H t

[0165] Where a is the smoothing parameter (the value of the smoothing parameter should be in the range of 0.1 to 0.7, determined by verifying its performance), H t The average structural entropy calculated for the latest round

[0166] The determined threshold is used as a criterion for determining whether the answer output by the model is hallucination or not;

[0167] By setting the threshold appropriately, we can effectively identify nodes or subgraphs with high semantic uncertainty, thus laying the foundation for subsequent labeling and processing.

[0168] S3.4.2. Output hallucination marking, including: identifying high entropy nodes, marking and recording, and rejecting output;

[0169] To identify high entropy nodes, the system automatically traverses each node and checks whether its structural entropy is higher than the set threshold;

[0170] Marking and recording: Mark the nodes that exceed the threshold and record relevant information, including: node characteristics, generated answer content, and structural entropy value;

[0171] Reject output means that for nodes marked as hallucination output, the system should reject the corresponding answer; this mechanism can effectively reduce the misleading information received by users, thereby improving the accuracy of the overall output;

[0172] Hallucination Mark Results S H, contains all answers marked as hallucinated output;

[0173] This process not only helps improve the user experience immediately, but also provides feedback for subsequent model training, helping developers identify and correct potential problems;

[0174] S3.5, such as Figure 5 As shown, the correct answer is output based on the hallucination mark; the steps are as follows:

[0175] S3.5.1, Eliminate hallucination output; According to the hallucination tag set S H , remove all answers marked as hallucination output from the answer set A to form a clean answer set A clean , the expression is as follows:

[0176] A clean =A / S H ;

[0177] S3.5.2, answer regeneration;

[0178] Specifically, for the input problem marked as hallucination, if A clean The answers in the query are insufficient to answer the original question, or too many answers are removed, resulting in incomplete information. The model needs to regenerate new answers.

[0179] The regeneration step includes: re-inputting the original question into the model and outputting the labeled hallucination S H Provided as negative samples to the model to help the model avoid generating similar hallucinated answers and for the regenerated answers, repeat embedding similarity detection and structural entropy calculation to ensure that the newly generated answers meet the set entropy threshold requirements;

[0180] S3.5.3. Output the final answer;

[0181] Comprehensive A clean and the new answers obtained through the regeneration process to form the final answer set A final , ensuring that all answers are not marked as hallucination outputs and meet semantic consistency and accuracy, and finally output the correct answer set A final .

[0182] This paper proposes a hallucination detection method based on semantic embedding and structural entropy detection, aiming to improve the performance of a medical question-answering system for esophageal cancer. Through semantic clustering and precise entropy calculation, this method effectively identifies and detects hallucination information in model outputs. The application of semantic embedding similarity detection avoids hallucination interference caused by traditional methods that rely on the model itself. The introduction of structural entropy further improves the accuracy of identifying semantically equivalent groups, making the model more reliable when processing complex medical information.

[0183] This approach has demonstrated excellent performance in question-answering and generative dialogue tasks, significantly improving the reliability and security of output. For example, in generating diagnostic recommendations, structural entropy helps identify recommendations with high uncertainty, thereby filtering out potential errors. Furthermore, in treatment recommendation, semantic clustering ensures that recommended options are scientifically sound and coherent.

[0184] With these improvements, the esophageal cancer model has become more reliable in providing medical advice and supporting patient decision-making. This approach not only enhances the model's practicality but also increases the trust of patients and medical professionals in the automated system, making its application in the medical field more widespread and effective.

Claims

1. A hallucination detection method based on semantic embedding and structural entropy detection, characterized in that: The following steps are involved: S1. Collect data on esophageal cancer; The data on esophageal cancer include: clinical data, database data and real case data; S2. Preprocess the collected data; The preprocessing includes: data cleaning and data standardization; S3, perform hallucination detection on the processed data and output the final answer; The steps of performing hallucination detection on the processed data and outputting the final answer are as follows: S3.

1. Input the processed data into a large language model for training. S3.

2. Similarity graph between answers generated by constructing the trained model; The specific steps of constructing the similarity graph between answers generated by the trained model are as follows: S3.2.

1. Input the question into the trained model and generate multiple corresponding answers and collect answers The answer set A is formed, and the expression is as follows: , i depends on the number of problems, where i∈(1,N), N is a natural integer; S3.2.2, using semantic embedding detection method to generate semantic similarity between answers; The semantic embedding detection method uses a sentence vector extraction method based on the BERT model to embed each generated answer Convert to embedding vector ; The conversion expression is as follows: ； The semantic similarity is calculated by using cosine similarity; The expression of cosine similarity is as follows: , i depends on the number of questions; i, j∈(1,N), N is a natural integer, i≠j Where, Represents the dot product of vectors; is the norm of the vector; S3.2.

3. Construct a similarity graph using the cosine similarity results. The construction process is to add a weighted edge to the undirected graph G; The expression of the undirected graph G is as follows: ； Where, is a vertex set, i.e. each generated answer corresponds to a vertex in the graph; It is a set of edges, which is calculated by calculating the cosine similarity between the answers; The expression for adding an edge with weight is as follows: is the cosine similarity value; the expression is as follows: ； ； Where, is the weight; The threshold is set based on empirical statistics; S3.

3. Calculating structural entropy based on the similarity graph; The expression for calculating the structural entropy based on the similarity graph is as follows: ； Where, is the structural entropy; Is with the node The sum of the weights of the connected edges; is the sum of the weights of all edges in an undirected graph; is a node The number of vertices in is a node The number of vertices in the parent node S3.4, Detecting hallucinations by optimizing structural entropy; The steps of detecting hallucinations by optimizing structural entropy are as follows: S3.4.

1. Obtain a threshold value through structural entropy, and perform hallucination detection based on the obtained threshold value; The expression of the threshold is as follows: ； Where, is the smoothing parameter; The average structural entropy calculated for the latest round ; is the initial threshold with flexibility when i=t , the expression is as follows: ； Where, The number of questions The average structural entropy of is the average value; The expression of the initial threshold is as follows: ； Where, To adjust the parameters, take 1.5; is the standard deviation; S3.4.

2. Output hallucination marking, including: identifying high entropy nodes, marking and recording, and rejecting output; The system automatically traverses each node to check whether its structural entropy is higher than a set threshold. The marking and recording not only marks the nodes exceeding the threshold, but also records relevant information, including: the characteristics of the node, the generated answer content, and the structural entropy value; The rejection output means that for the node marked as hallucination output, the system rejects its corresponding answer; S3.5, give the correct answer based on the hallucination mark; The steps of giving the correct answer based on hallucination marks are as follows: S3.5.1, Eliminate hallucination output; Remove all answers marked as hallucination outputs from the answer set A to form a clean answer set , the expression is as follows: ； Where, Mark collections for illusions; S3.5.

2. Answer regeneration, including: re-inputting the original question into the model and outputting the labeled hallucinations Provided to the model as negative samples; S3.5.

3. Output the final answer .

2. A hallucination detection device based on semantic embedding and structural entropy detection, characterized in that: The device is used to execute the hallucination detection method based on semantic embedding and structural entropy detection described in claim 1.

Images