Illusion detection method and device based on semantic embedding and structural entropy detection

Abstract

The invention provides an illusion detection method and device based on semantic embedding and structure entropy detection. The method comprises the following steps: aiming at an input question, generating a plurality of possible answer sets by utilizing a trained language model; converting each answer into a vector by using a pre-trained sentence embedding model, and calculating cosine similarity between the answers so as to construct a similarity graph; representing the similarity graph through a coding tree, and calculating the structure entropy of each node to identify nodes with relatively high semantic uncertainty; dynamically adjusting a threshold value by adopting a moving average method to ensure that high-entropy output is marked as illusion in time; and the answers marked as illusion are eliminated, so that the accuracy of the output answers and the information consistency are ensured. According to the method, the performance of the model on complex problems is effectively improved, and the generation of misleading information is reduced. Preliminary experiments show that according to the method, the illusion output proportion is remarkably reduced on multiple data sets, and the semantic consistency and accuracy of generated answers are enhanced.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing, and in particular to a hallucination detection method and a device based on semantic embedding and structural entropy detection. Background Art

[0002] In the medical field, especially in the scenarios of disease diagnosis and treatment consultation, the accuracy of information is crucial. For example, in the diagnosis and treatment recommendations of esophageal cancer, incorrect information may not only mislead patients, but also affect their life safety. Therefore, ensuring the reliability and accuracy of content generated by large language models has become an important topic of current research. With the rapid development of artificial intelligence technology, significant progress has been made in the field of natural language processing (NLP). Large language models (LLMs) are widely used in various tasks such as question-answering systems, generative dialogues, text summarization, and machine translation due to their powerful text generation and understanding capabilities. However, these models still face the risk of false or inaccurate output when generating content, especially when it comes to sensitive areas such as medical, legal, and financial, this risk may lead to serious consequences.

[0003] Existing methods still have certain limitations in practical applications, and it is difficult to fully solve the problem of false information detection. For this reason, it is particularly important to develop a new hallucination detection system that can integrate semantic embedding and structural entropy. This innovation can not only improve the performance of large language models in medical question-answering, but also reduce the risk of false output in high-risk fields such as medicine, thereby providing users with safer and more reliable services. In order to solve this problem, researchers have begun to explore a variety of technical means. As an important concept in information theory, structural entropy effectively measures the distribution and uncertainty of information, and helps identify potential false outputs by analyzing the information structure in the text. At the same time, semantic embedding technology further improves the credibility of content by analyzing the semantic similarity between generated content and real data. The combination of these technologies provides new ideas for improving the reliability of large language models. Summary of the invention

[0004] To solve the above problems, the present invention provides a hallucination detection method based on semantic embedding and structural entropy detection.

[0005] To implement the above technical solution, the specific steps are as follows:

[0006] A hallucination detection method based on fusion of semantic embedding and structural entropy includes the following steps:

[0007] S1. Collect data related to esophageal cancer. Through multi-dimensional data collection strategies, it provides a rich and reliable data basis for subsequent hallucination detection, ensuring that the system's evaluation in esophageal cancer detection is more scientific and effective.

[0008] In order to effectively evaluate the performance of large language models in esophageal cancer detection, data collection is crucial; this process aims to obtain relevant questions and answers to analyze the accuracy and reliability of the content generated by the model;

[0009] The collected data include: clinical data, database data and real case data;

[0010] Clinical data include: clinical questions and standard answers; clinical questions are collected from actual problems encountered by patients and doctors, such as "What are the early symptoms of esophageal cancer?" and "What are the common treatment methods?"; standard answers are collected from authoritative medical literature and clinical practice.

[0011] The database includes: medical literature, clinical guidelines and online Q&A platform; medical literature refers to the research on esophageal cancer by consulting relevant academic papers and systematic reviews to obtain corresponding solutions; clinical guidelines refer to the information on esophageal cancer research extracted from the clinical guidelines issued by the American Cancer Society; the online Q&A platform uses HealthTap and WebMD to collect interactive Q&A between esophageal cancer patients and doctors to reflect real clinical questions;

[0012] Real case data refers to the collection of symptom descriptions and diagnosis results of patients diagnosed with esophageal cancer, including: medical records from medical institutions and case reports in professional journals;

[0013] These data provide the symptoms and diversity of real cases and help frame relevant questions;

[0014] S2, preprocessing the collected data;

[0015] Data preprocessing is a critical step to ensure that the answers generated by the model are consistent and accurate with the clinical questions; it includes the following steps:

[0016] S2.1. Data cleaning: The goal of the data cleaning stage is to remove noise and inconsistencies in the collected data to ensure the purity and availability of the data. The specific steps are as follows:

[0017] S2.1.1. Remove redundant characters and potential interference information to make the data easier to analyze;

[0018] Specifically, removing redundant characters means cleaning up special characters and irrelevant punctuation marks in text data to ensure the neatness of text content;

[0019] S2.1.2. Correct spelling errors; accurate text input can improve the learning effect of subsequent models;

[0020] Specifically, use spell-checking tools to identify and correct spelling errors in text to enhance the readability and accuracy of data;

[0021] S2.1.3. Processing missing values ​​ensures data integrity and avoids analysis bias caused by missing information;

[0022] Specifically, by manually identifying missing clinical questions or answers in the collected data, it is decided whether to fill in, delete, or keep them as they are to ensure the integrity of the data set;

[0023] S2.2. Data standardization: convert the cleaned data into a unified format to facilitate subsequent analysis and input; the specific steps are as follows:

[0024] S2.2.1. Text normalization makes the model more efficient in processing text;

[0025] Specifically, all text data is converted to lowercase to reduce the impact of case differences on analysis, and redundant spaces and line breaks are removed to ensure consistency in text format;

[0026] S2.2.2. Numerical standardization: By converting numerical data to the same scale, the training effect of the model is improved to ensure that each feature is equally important in model learning;

[0027] Specifically, the numerical data related to the patient's symptoms (such as age and tumor size) were standardized to eliminate the influence of different dimensions on the analysis;

[0028] Through data preprocessing, a solid data foundation was established to support the accuracy and reliability of the model in esophageal cancer detection, and the processed data was used to determine the hyperparameters;

[0029] S3, perform hallucination detection on the processed data and output the final answer; the steps are as follows:

[0030] S3.1. Input the processed data into a large language model (GPT-4) to train the model;

[0031] S3.2. Construct a similarity graph between the answers generated by the trained model. The generated similarity graph is used for the calculation of structural entropy later. The steps are as follows:

[0032] S3.2.1. Input the question into the trained model and generate multiple corresponding answers s (i) and collect answers (i) The answer set A is formed, and the expression is as follows:

[0033] A=s (i) , i depends on the number of problems, where i∈(1,N), N is a natural integer;

[0034] S3.2.2, using semantic embedding detection method to generate semantic similarity between answers;

[0035] Specifically, we use the sentence vector extraction method based on the BERT model (Sentence-BERT) to extract each generated answer s (i) Convert to embedding vector e (i) , and use cosine similarity to calculate semantic similarity;

[0036] The conversion expression is as follows:

[0037] e (i) =SBERT(s (i) );

[0038] The expression of cosine similarity is as follows:

[0039] iDepends on the number of questions;

[0040] i, j∈(1, N), N is a natural integer, i≠j

[0041] In the formula, e (i) ·e (j) Represents the dot product of vectors, which can reflect the similar directions of two vectors; ‖e (i) ‖‖e (j) ‖ is the norm of the vector, which ensures that the similarity value is between 0 and 1. The closer the value is to 1, the more similar it is.

[0042] S3.2.3, construct a similarity graph using the cosine similarity results;

[0043] Specifically, a similarity graph is obtained by adding a weighted edge to the undirected graph G;

[0044] The expression of the undirected graph G is as follows:

[0045] G=(V,E)

[0046] Where V is the vertex set, i.e. each generated answer s (i) Corresponding to a vertex in the graph; E is the edge set, which is calculated by the cosine similarity between the answers and adds edges according to the cosine similarity. The edge weights between answers with high cosine similarity are higher, and vice versa;

[0047] Furthermore, the threshold θ is set based on empirical statistics. b , used to determine whether two answers are similar enough to establish a connection; for each pair of answers s (i) and (j) , if its similarity is higher than the threshold θ b , then add a weighted edge to the undirected graph G, the weight is the cosine similarity value; the expression is as follows:

[0048] if sim(s (i) ,s (j) )>θ b ;

[0049] G.add_edge(s (i) ,s (j) ,weight=sim(s (i) ,s (j) ))

[0050] In the formula, weight is the weight;

[0051] S3.3. Calculate the structural entropy based on the similarity graph; the expression is as follows:

[0052]

[0053] In the formula, H T (G; α) is the structural entropy; g α is the sum of the weights of the edges connected to node α (the generated answer or the result of a semantic clustering); m is the sum of the weights of all edges in the undirected graph; V α is the number of vertices in node α; V α - is the number of vertices in the parent node of node α;

[0054] Furthermore, the coding tree T is a tree structure representation of the undirected graph G, and each node of the coding tree corresponds to a non-empty vertex subset in V; the root node represents the entire vertex set of the undirected graph G; each leaf node corresponds to a vertex in the undirected graph G, that is, a generated answer;

[0055] The total structural entropy of the undirected graph G is the sum of the entropies of all nodes in the coding tree T; the expression is as follows:

[0056] H T (G) = ∑ α∈T H T (G; α);

[0057] By minimizing structural entropy, the system is able to identify semantically equivalent groups of generated answers;

[0058] S3.4, Detecting hallucinations by optimizing structural entropy;

[0059] Specifically, by presetting a threshold, all nodes or subgraphs whose structural entropy values ​​exceed the threshold are marked as hallucination outputs, and these nodes and their corresponding generated answers are recorded to form a hallucination output set S H ; The specific steps are as follows:

[0060] S3.4.1, Optimize structural entropy;

[0061] After calculating the structural entropy of each node or subgraph, the next step is to set a reasonable threshold. The choice of this threshold is crucial, as it not only affects the detection effect of hallucination output, but also directly affects the overall performance of the system. When setting the threshold, three factors need to be considered, including: empirical basis, data characteristics, and dynamic adjustment.

[0062] The empirical basis is to select a threshold that can effectively distinguish normal and abnormal outputs based on past experience and experimental evidence. It is necessary to perform statistics on the structural entropy distribution on multiple data sets and determine its average value. and standard deviation σ H ; Initial threshold θ u Can be set to:

[0063]

[0064] Wherein, k is an adjustment parameter, which usually ranges from 1.0 to 2.0. In this embodiment, in order to be as comprehensive as possible and avoid mislabeling, it is set to 1.5;

[0065] Then, by selecting a threshold suitable for a specific task and data, we analyze the data characteristics and the diversity and complexity of the generated answers. Different types of questions may lead to different structural entropy distributions, so the setting of the initial threshold should be flexible. The initial threshold is further designed and expressed as follows:

[0066]

[0067] In the formula, is the average structural entropy of the number of problems i;

[0068] Dynamic adjustment means that in actual applications, the threshold is flexibly and dynamically adjusted according to real-time feedback and system performance. For example, if the system finds that some high entropy outputs are often marked as hallucination outputs and affect the user experience, it is necessary to use the sliding average method to mark the flexible initial threshold. The flexible initial threshold is further designed, and the expression is as follows:

[0069] θ t+1 =(1-a)·θ t +a·H t

[0070] Where a is the smoothing parameter (the value of the smoothing parameter should be in the range of 0.1 to 0.7, determined by verifying its performance), H t The average structural entropy value calculated for the latest round

[0071] The determined threshold is used as a criterion for determining whether the answer output by the model is an illusion or not;

[0072] By setting the threshold appropriately, nodes or subgraphs with high semantic uncertainty can be effectively identified, thus laying the foundation for subsequent labeling and processing;

[0073] S3.4.2. Output hallucination marking, including: identifying high entropy nodes, marking and recording, and rejecting output;

[0074] To identify high entropy nodes, the system automatically traverses each node to check whether its structural entropy is higher than the set threshold;

[0075] Marking and recording: Mark the nodes that exceed the threshold and record relevant information, including: node characteristics, generated answer content, and structural entropy value;

[0076] Reject output means that for nodes marked as hallucination output, the system should reject the corresponding answer; this mechanism can effectively reduce the misleading information received by users, thereby improving the accuracy of the overall output;

[0077] Hallucination Mark Results H , contains all answers marked as hallucinated output;

[0078] S3.5. Output the correct answer based on the hallucination mark; the steps are as follows:

[0079] S3.5.1. Eliminate hallucination output; according to the hallucination tag set S H , remove all answers marked as hallucination output from the answer set A to form a clean answer set A clean , the expression is as follows:

[0080] A clean =A / S H ;

[0081] S3.5.2, answer regeneration;

[0082] Specifically, for the input problem marked as hallucination, if A clean The answers in the query are not sufficient to answer the original question, or too many answers are removed, resulting in incomplete information. The model needs to regenerate new answers.

[0083] The regeneration step includes: re-inputting the original question into the model and outputting the labeled hallucination S H Provided as negative samples to the model to help the model avoid generating similar hallucinated answers and for the regenerated answers, repeat embedding similarity detection and structural entropy calculation to ensure that the newly generated answers meet the set entropy threshold requirements;

[0084] S3.5.3, output the final answer;

[0085] Comprehensive A cleanand the new answers obtained through the regeneration process to form the final answer set A final , ensuring that all answers are not marked as hallucination outputs and meet semantic consistency and accuracy, and finally output the correct answer set A final .

[0086] Beneficial effects of the present invention

[0087] 1. The present invention significantly improves the reliability and accuracy of the medical question-answering system related to esophageal cancer by integrating semantic embedding and structural entropy detection.

[0088] 2. The present invention intuitively displays the relationship and similarity between different treatment plans by integrating graph structure and entropy calculation, providing more personalized and accurate medical services, while effectively detecting and reducing the risk of the model generating hallucination information, thereby improving the accuracy of diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] Figure 1 is a flow chart of the present invention;

[0090] Figure 2 It is the overall framework diagram of the present invention;

[0091] Figure 3 Data collection and preprocessing for the present invention;

[0092] Figure 4 is a flow chart of hallucination detection of the present invention;

[0093] Figure 5 It is a flowchart of discarding illusion answers of the present invention. DETAILED DESCRIPTION

[0094] The present invention is further described in detail below in conjunction with specific embodiments.

[0095] like Figure 1-2 As shown, a hallucination detection method based on fusion of semantic embedding and structural entropy includes the following steps:

[0096] S1. Collect data related to esophageal cancer. Through multi-dimensional data collection strategies, it provides a rich and reliable data basis for subsequent hallucination detection, ensuring that the system's evaluation in esophageal cancer detection is more scientific and effective.

[0097] In order to effectively evaluate the performance of large language models in esophageal cancer detection, data collection is crucial; this process aims to obtain relevant questions and answers to analyze the accuracy and reliability of the content generated by the model;

[0098] The collected data include: clinical data, database data and real case data;

[0099] Clinical data include: clinical questions and standard answers; clinical questions are collected from actual problems encountered by patients and doctors, such as "What are the early symptoms of esophageal cancer?" and "What are the common treatment methods?"; standard answers are collected from authoritative medical literature and clinical practice.

[0100] The database includes: medical literature, clinical guidelines and online Q&A platform; medical literature refers to the research on esophageal cancer by consulting relevant academic papers and systematic reviews to obtain corresponding solutions; clinical guidelines refer to the information on esophageal cancer research extracted from the clinical guidelines issued by the American Cancer Society; the online Q&A platform uses HealthTap and WebMD to collect interactive Q&A between esophageal cancer patients and doctors to reflect real clinical questions;

[0101] Real case data refers to the collection of symptom descriptions and diagnosis results of patients diagnosed with esophageal cancer, including: medical records from medical institutions and case reports in professional journals;

[0102] These data provide the symptoms and diversity of real cases and help frame relevant questions;

[0103] S2, such as Figure 3 As shown, the collected data is preprocessed; data preprocessing is a key step to ensure that the answers generated by the model are consistent and accurate with the clinical questions; it includes the following steps:

[0104] S2.1. Data cleaning: The goal of the data cleaning stage is to remove noise and inconsistencies in the collected data to ensure the purity and availability of the data. The specific steps are as follows:

[0105] S2.1.1. Remove redundant characters and potential interference information to make the data easier to analyze;

[0106] Specifically, removing redundant characters means cleaning up special characters and irrelevant punctuation marks in text data to ensure the neatness of text content;

[0107] S2.1.2. Correct spelling errors; accurate text input can improve the learning effect of subsequent models;

[0108] Specifically, use spell-checking tools to identify and correct spelling errors in text to enhance the readability and accuracy of data;

[0109] S2.1.3. Processing missing values ​​ensures data integrity and avoids analysis bias caused by missing information;

[0110] Specifically, by manually identifying missing clinical questions or answers in the collected data, it is decided whether to fill in, delete, or keep them as they are to ensure the integrity of the data set;

[0111] S2.2. Data standardization: convert the cleaned data into a unified format to facilitate subsequent analysis and input; the specific steps are as follows:

[0112] S2.2.1. Text normalization makes the model more efficient in processing text;

[0113] Specifically, all text data is converted to lowercase to reduce the impact of case differences on analysis, and redundant spaces and line breaks are removed to ensure consistency in text format;

[0114] S2.2.2. Numerical standardization: By converting numerical data to the same scale, the training effect of the model is improved to ensure that each feature is equally important in model learning;

[0115] Specifically, the numerical data related to the patient's symptoms (such as age and tumor size) were standardized to eliminate the influence of different dimensions on the analysis;

[0116] Through data preprocessing, a solid data foundation was established to support the accuracy and reliability of the model in esophageal cancer detection, and the processed data was used to determine the hyperparameters;

[0117] S3, such as Figure 4 As shown, hallucination detection is performed on the processed data and the final answer is output; the steps are as follows:

[0118] S3.1. Input the processed data into a large language model (GPT-4) to train the model;

[0119] S3.2. Construct a similarity graph between the answers generated by the trained model. The generated similarity graph is used for the calculation of structural entropy later. The steps are as follows:

[0120] S3.2.1. Input the question into the trained model and generate multiple corresponding answers s (i) and collect answers (i) The answer set A is formed, and the expression is as follows:

[0121] A=s (i) , i depends on the number of problems, where i∈(1,N), N is a natural integer;

[0122] These generated answers are used for semantic clustering and structural entropy calculation, by comparing and clustering the answers based on semantic similarity in order to discover potential themes or meaning clusters;

[0123] In addition, structural entropy calculation is used to evaluate the diversity and information content of the generated answers. By calculating the structural entropy of the answers, researchers can understand the complexity and information richness of the answers, and further analyze the diversity and quality of the answers generated by the model.

[0124] The comprehensive use of the probability distribution and semantic clustering of these generated answers can help us deeply understand the response of the model to specific input questions and the characteristics of the generated answers, providing more useful information for subsequent applications and analysis;

[0125] S3.2.2, using semantic embedding detection method to generate semantic similarity between answers;

[0126] The semantic embedding detection method can effectively capture the semantic similarity between generated answers, helping users to deeply analyze and understand the content generated by the model. This method not only improves the clustering effect of the answers, but also provides rich semantic information for subsequent applications.

[0127] Specifically, we use the sentence vector extraction method based on the BERT model (Sentence-BERT) to extract each generated answer s (i) Convert to embedding vector e (i) , and use cosine similarity to calculate semantic similarity;

[0128] The conversion expression is as follows:

[0129] e (i) =SBERT(s (i) );

[0130] The expression of cosine similarity is as follows:

[0131] iDepends on the number of questions;

[0132] i, j∈(1, N), N is a natural integer, i≠j

[0133] In the formula, e (i) ·e (j) Represents the dot product of vectors, which can reflect the similar directions of two vectors; ||e (i) ||||e (j) || is the norm of the vector, which ensures that the similarity value is between 0 and 1. The closer the value is to 1, the more similar it is.

[0134] S3.2.3, construct a similarity graph using the cosine similarity results;

[0135] Specifically, a similarity graph is obtained by adding a weighted edge to the undirected graph G;

[0136] The expression of the undirected graph G is as follows:

[0137] G=(V,E)

[0138] Where V is the vertex set, i.e. each generated answer s (i)Corresponding to a vertex in the graph; E is the edge set, which is calculated by the cosine similarity between the answers and adds edges according to the cosine similarity. The edge weights between answers with high cosine similarity are higher, and vice versa;

[0139] Furthermore, the threshold θ is set based on empirical statistics. b , used to determine whether two answers are similar enough to establish a connection; for each pair of answers s (i) and (j) , if its similarity is higher than the threshold θ b , then add a weighted edge to the undirected graph G, the weight is the cosine similarity value; the expression is as follows:

[0140] if sim(s (i) ,s (j) )>θ b ;

[0141] G.add_edge(s (i) ,s (j) ,weight=sim(s (i) ,s (j) ))

[0142] In the formula, weight is the weight;

[0143] S3.3. Calculate the structural entropy based on the similarity graph. Structural entropy is an indicator that measures the complexity and uncertainty of information and can reveal the semantic diversity of generated answers. The expression is as follows:

[0144]

[0145] In the formula, H T (G; α) is the structural entropy; g α is the sum of the weights of the edges connected to node α (the generated answer or the result of a semantic clustering); m is the sum of the weights of all edges in the undirected graph; V α is the number of vertices in node α; V α - is the number of vertices in the parent node of node α;

[0146] Furthermore, the coding tree T is a tree structure representation of the undirected graph G, and each node of the coding tree corresponds to a non-empty vertex subset in V; the root node represents the entire vertex set of the undirected graph G; each leaf node corresponds to a vertex in the undirected graph G, that is, a generated answer;

[0147] In this embodiment, the root node represents the set of all possible answers related to esophageal cancer; the leaf nodes represent the specific answers actually generated; the vertices represent specific topics, question types, or similar answer sets;

[0148] This tree-structured representation can better organize and display the relationships and hierarchical structures between generated answers. The construction of the encoding tree can help users understand the hierarchical relationships between generated answers, gradually showing the connections between answers from the overall to the details, which helps to more comprehensively analyze and understand the structure and semantic features of generated answers.

[0149] The total structural entropy of the undirected graph G is the sum of the entropies of all nodes in the coding tree T; the expression is as follows:

[0150] H T (G) = ∑ α∈T H T (G; α);

[0151] By minimizing structural entropy, the system is able to identify semantically equivalent groups of generated answers;

[0152] S3.4, Detecting hallucinations by optimizing structural entropy;

[0153] According to the constructed similarity graph, the structural information of the similarity graph can be effectively used to calculate the structural entropy, and the hallucination output can be detected accordingly;

[0154] Specifically, by presetting a threshold, all nodes or subgraphs whose structural entropy values ​​exceed the threshold are marked as hallucination outputs, and these nodes and their corresponding generated answers are recorded to form a hallucination output set S H ; The specific steps are as follows:

[0155] S3.4.1, Optimize structural entropy;

[0156] After calculating the structural entropy of each node or subgraph, the next step is to set a reasonable threshold. The choice of this threshold is crucial, as it not only affects the detection effect of hallucination output, but also directly affects the overall performance of the system. When setting the threshold, three factors need to be considered, including: empirical basis, data characteristics, and dynamic adjustment.

[0157] The empirical basis is to select a threshold that can effectively distinguish normal and abnormal outputs based on past experience and experimental evidence. It is necessary to perform statistics on the structural entropy distribution on multiple data sets and determine its average value. and standard deviation σ H ; Initial threshold θ u Can be set to:

[0158]

[0159] Wherein, k is an adjustment parameter, which usually ranges from 1.0 to 2.0. In this embodiment, in order to be as comprehensive as possible and avoid mislabeling, it is set to 1.5;

[0160] Then, by selecting a threshold suitable for a specific task and data, we analyze the data characteristics and the diversity and complexity of the generated answers. Different types of questions may lead to different structural entropy distributions, so the setting of the initial threshold should be flexible. The initial threshold is further designed and expressed as follows:

[0161]

[0162] In the formula, is the average structural entropy of the number of problems i;

[0163] Dynamic adjustment means that in actual applications, the threshold is flexibly and dynamically adjusted according to real-time feedback and system performance. For example, if the system finds that some high entropy outputs are often marked as hallucination outputs and affect the user experience, it is necessary to use the sliding average method to mark the flexible initial threshold. The flexible initial threshold is further designed, and the expression is as follows:

[0164] θ t+1 =(1-a)·θ t +a·H t

[0165] Where a is the smoothing parameter (the value of the smoothing parameter should be in the range of 0.1 to 0.7, determined by verifying its performance), H t The average structural entropy value calculated for the latest round

[0166] The determined threshold is used as a criterion for determining whether the answer output by the model is an illusion or not;

[0167] By setting the threshold appropriately, nodes or subgraphs with high semantic uncertainty can be effectively identified, thus laying the foundation for subsequent labeling and processing;

[0168] S3.4.2. Output hallucination marking, including: identifying high entropy nodes, marking and recording, and rejecting output;

[0169] To identify high entropy nodes, the system automatically traverses each node to check whether its structural entropy is higher than the set threshold;

[0170] Marking and recording: Mark the nodes that exceed the threshold and record relevant information, including: node characteristics, generated answer content, and structural entropy value;

[0171] Reject output means that for nodes marked as hallucination output, the system should reject the corresponding answer; this mechanism can effectively reduce the misleading information received by users, thereby improving the accuracy of the overall output;

[0172] Hallucination Mark Results H, contains all answers marked as hallucinated output;

[0173] This process not only helps improve the user experience immediately, but also provides feedback for subsequent model training, helping developers identify and correct potential problems;

[0174] S3.5, if Figure 5 As shown, the correct answer is output based on the hallucination mark; the steps are as follows:

[0175] S3.5.1. Eliminate hallucination output; according to the hallucination tag set S H , remove all answers marked as hallucination output from the answer set A to form a clean answer set A clean , the expression is as follows:

[0176] A clean =A / S H ;

[0177] S3.5.2, answer regeneration;

[0178] Specifically, for the input problem marked as hallucination, if A clean The answers in the query are not sufficient to answer the original question, or too many answers are removed, resulting in incomplete information. The model needs to regenerate new answers.

[0179] The regeneration step includes: re-inputting the original question into the model and outputting the labeled hallucination S H Provided as negative samples to the model to help the model avoid generating similar hallucinated answers and for the regenerated answers, repeat embedding similarity detection and structural entropy calculation to ensure that the newly generated answers meet the set entropy threshold requirements;

[0180] S3.5.3, output the final answer;

[0181] Comprehensive A clean and the new answers obtained through the regeneration process to form the final answer set A final , ensuring that all answers are not marked as hallucination outputs and meet semantic consistency and accuracy, and finally output the correct answer set A final .

[0182] The present invention proposes a hallucination detection method based on semantic embedding and structural entropy detection, aiming to improve the performance of the medical question-answering system related to esophageal cancer. Through semantic clustering and precise entropy calculation, this method can effectively identify and detect hallucination information in the model output. The application of semantic embedding similarity detection avoids the hallucination interference caused by the reliance on the model itself in traditional methods. The introduction of structural entropy further improves the recognition accuracy of semantically equivalent groups, making the model more reliable when processing complex medical information.

[0183] This method performs well in question-answering and generative dialogue tasks, significantly improving the reliability and safety of the output. For example, in the process of generating diagnostic suggestions, structural entropy helps identify suggestions with high uncertainty, thereby filtering out potential erroneous information. In addition, in treatment plan recommendations, semantic clustering ensures that the recommended plans are scientifically based and coherent.

[0184] With these improvements, the esophageal cancer model becomes more credible in providing medical advice and supporting patient decision-making. This approach not only enhances the usefulness of the model, but also increases the trust of patients and medical professionals in the automated system, making its application in the medical field more widespread and effective.

Claims

1. A hallucination detection method based on semantic embedding and structural entropy detection, characterized in that: The following steps are involved: S1. Collect data on esophageal cancer; The data on esophageal cancer include: clinical data, database data and real case data; S2, preprocessing the collected data; The preprocessing includes: data cleaning and data standardization; S3. Perform hallucination detection on the processed data and output the final answer.

2. The method for hallucination detection based on semantic embedding and structural entropy detection according to claim 1, characterized in that: The steps of performing hallucination detection on the processed data and outputting the final answer are as follows: S3.

1. Input the processed data into a large language model to train the model; S3.2, by constructing a similarity graph between answers generated by the trained model; S3.3, calculating structural entropy based on the similarity graph; S3.4, Detecting hallucinations by optimizing structural entropy; S3.

5. Give the correct answer based on the hallucination mark.

3. The method for hallucination detection based on semantic embedding and structural entropy detection according to claim 2, characterized in that: The specific steps of constructing a similarity graph between answers generated by a trained model are as follows: S3.2.

1. Input the question into the trained model and generate multiple corresponding answers s( i ) and collect the answers s( i ) to form the answer set A, which is expressed as follows: A=s( i ), i depends on the number of problems, where i∈(1,N), N is a natural integer; S3.2.2, using semantic embedding detection method to generate semantic similarity between answers; The semantic embedding detection method uses a sentence vector extraction method based on the BERT model to embed each generated answer s( i ) is converted to an embedding vector e( i ); The conversion expression is as follows: e(i) = SBERT(s(i)); The semantic similarity is calculated by using cosine similarity; The expression of cosine similarity is as follows: iDepends on the number of questions; i, j∈(1,N), N is a natural integer, i≠j Where, e(i)·e(j) represents the dot product of the vectors; ||e(i)||||e(j)|| is the norm of the vector; S3.2.3, construct a similarity graph using the cosine similarity results; The construction process is to add a weighted edge to the undirected graph G; The expression of the undirected graph G is as follows: G=(V,E) Where V is the vertex set, that is, each generated answer s(i) corresponds to a vertex in the graph; E is the edge set, that is, the cosine similarity between the answers is calculated; The expression for adding an edge with weight is as follows: is the cosine similarity value; the expression is as follows: ifsim(s(i),s0j))>θbb; G.add_edge(s(i)),s(j),weight=sim(si),s(j))) In the formula, weight is the weight; θb is the threshold set according to empirical statistics.

4. The method for hallucination detection based on semantic embedding and structural entropy detection according to claim 2, characterized in that: The expression for calculating the structural entropy based on the similarity graph is as follows: In the formula, H T (G; α) is the structural entropy; g α is the sum of the weights of the edges connected to node α; m is the sum of the weights of all edges in the undirected graph; V α is the number of vertices in node α; V α - is the number of vertices in the parent node of node α.

5. The method for hallucination detection based on semantic embedding and structural entropy detection according to claim 2, characterized in that: The steps of detecting hallucinations by optimizing structural entropy are as follows: S3.4.1, obtaining a threshold value through structural entropy, and performing hallucination detection based on the obtained threshold value; The expression of the threshold is as follows: θt+1=(1-a)·θt+a·Ht Where a is the smoothing parameter; Ht is the average structural entropy value of the latest round of calculations θt is the initial threshold θi with flexibility when i=t, and the expression is as follows: In the formula, is the average structural entropy of the number of problems i; is the average value; The expression of the initial threshold is as follows: In the formula, k is the adjustment parameter, which is 1.5; σ H is the standard deviation; S3.4.

2. Output hallucination marking, including: identifying high entropy nodes, marking and recording, and rejecting output; The high entropy node identification system automatically traverses each node to check whether its structural entropy is higher than a set threshold; The marking and recording not only marks the nodes exceeding the threshold, but also records the relevant information, including: the characteristics of the nodes, the generated answer content and the structural entropy value; The rejection output is that for the node marked as hallucination output, the system rejects its corresponding answer.

6. The method for hallucination detection based on semantic embedding and structural entropy detection according to claim 1, characterized in that: The steps for giving the correct answer based on hallucination marks are as follows: S3.5.1, Eliminate hallucination output; Remove all answers marked as hallucination outputs from answer set A to form a clean answer set A. clean , the expression is as follows: A clean =A / S H ; In the formula, S H Marking collections for illusions; S3.5.

2. Answer regeneration, including: re-inputting the original question into the model and outputting the labeled hallucination S H Provided to the model as negative samples; S3.5.

3. Output the final answer A final .

7. A hallucination detection device based on semantic embedding and structural entropy detection, characterized in that: The device is used to execute the hallucination detection method based on semantic embedding and structural entropy detection as described in any one of claims 1 to 6.

Images