Large language model illusion detection method based on graph neural network
By constructing a weighted directed graph and utilizing graph neural networks to detect illusions in large language models, this method solves the problem of existing methods being unable to effectively combine token semantic features and dependencies, achieving efficient and interpretable illusion detection and improving the reliability of the model.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NAT UNIV OF DEFENSE TECH
- Filing Date
- 2025-12-06
- Publication Date
- 2026-05-08
AI Technical Summary
Existing large language models generate illusions in vertical domains due to a lack of relevant knowledge, leading to misleading user decisions and threatening applications in high-risk scenarios. Existing illusion detection methods cannot effectively combine the semantic features and dependencies of tokens and lack interpretability.
We construct a weighted directed graph, couple the hidden state and attention matrix of a large language model, use graph neural networks for hallucination detection, extract global graph representations through graph convolutional neural networks, and introduce interpretability analysis.
It achieves efficient and interpretable hallucination detection, improves the reliability and detection performance of large language models, and provides token-level interpretable analysis.
Smart Images
Figure CN121997112A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of classification and detection technology, and in particular relates to a method for hallucination detection based on a large language model using graph neural networks. Background Technology
[0002] Large language models (MLMs) have demonstrated exceptional capabilities in intent understanding and natural language generation, integrating multi-source information to directly generate responses, eliminating the tedious information integration process for users. These advantages have driven the widespread application of MLMs in fields such as information retrieval and cybersecurity. However, MLMs can also generate fluent but factually contradictory responses due to a lack of relevant knowledge—a phenomenon known as the "illusion" of large language models. Notably, this problem is particularly pronounced in vertical domains because MLMs typically require fine-tuning with specialized knowledge to improve the performance of downstream tasks, and using fine-tuned samples containing new knowledge exacerbates the illusion tendency. The illusionary behavior of large language models not only misleads user decisions but also poses a significant threat to applications in high-risk scenarios. Therefore, to ensure the reliability of large language models, it is essential to design relevant illusion detection mechanisms to proactively issue illusion warnings to users or the system when the content generated by the large language model deviates from reality.
[0003] Many works have proposed corresponding detection methods for the illusion phenomenon of large language models. Based on whether or not the internal state of the large language model is accessed, these methods can be roughly divided into black-box and white-box methods.
[0004] Black-box illusion detection methods are a form of post-hoc evaluation, primarily assessing the authenticity of content from a natural language perspective after responses are generated. For example, the SelfCheckGPT method queries a large language model multiple times for the same question and samples multiple responses, then calculates their consistency to determine the model's confidence in the responses. However, this method cannot avoid the inherent flaw of large language models—overconfidence—and multiple sampling incurs significant computational overhead. The HaluCheck method automates illusion detection through three sub-processes: fact decomposition, retrieval augmentation, and result comparison. While secondary verification using external resources can largely guarantee the authenticity of the detection results, requesting external resources for each response inevitably leads to unnecessary resource and time consumption. Furthermore, the updating and management of external resource repositories requires regular maintenance by industry experts.
[0005] White-box illusion detection methods offer real-time evaluation without requiring any external resources. By analyzing and learning the latent parameter space of a large language model, efficient illusion detectors are developed and designed. While achieving real-time detection, white-box methods strategically decide when to invoke external resources, thus optimizing latency and computational overhead in practical applications. The Perplexity method, a high-precision illusion detection method utilizing the logits output of the last layer of a large language model, demonstrates superior performance in summarizing and translation tasks, but it cannot assess and quantify the uncertainty of natural language. The Semantic Entropy method incorporates a measure of language invariance arising from shared meaning, proving that semantic entropy improves the accuracy of prediction models. The SAPLMA method achieves superior performance by extracting the internal representations of question-answer pairs and then training a fact classifier to determine whether the response contains illusions. This indicates that the internal state contains more hidden information about whether the large language model is lying. Compared to black-box methods, white-box methods, using the latent space of the large language model to distinguish between real and illusion responses, are more real-time and efficient. However, existing latent space-based methods either utilize hidden state alone or use attention matrix alone, which decouples the interrelationships between internal states, thus weakening the overall semantic space features. Considering that large language models incorporate both semantic features and learned connections and weights between tokens during pre-training, the decoupling methods in existing works cannot reference the correct patterns learned during pre-training, cannot effectively distinguish between real and illusory content, and lack interpretability. Summary of the Invention
[0006] To address the aforementioned technical issues and simultaneously model the semantic features of tokens and the dependencies between tokens, this invention proposes using a weighted directed graph to couple the hidden state and the attention matrix, and introduces a hallucination detection technology based on graph neural networks.
[0007] The first aspect of this invention discloses a method for hallucination detection based on a large language model using graph neural networks, the method comprising: Step S1: Construct a labeled sample set containing question-answer pairs; and couple the hidden state of a specific layer of the large language model with the attention matrix to establish a weighted directed graph. In these question-and-answer pairs, the answers are labeled as fact or illusion; Step S2: Input the weighted directed graph into the illusion detector based on graph neural network for supervised learning and training; Step S3: Deploy the hallucination detector that has completed supervised learning and training, perform real-time hallucination detection on question-answer pairs, and simultaneously provide token-level interpretability analysis.
[0008] In step S1, a weighted directed graph is established, wherein: the weighted directed graph is used to represent the hidden state and attention matrix of a specific layer of the coupled large language model, the nodes in the graph represent the tokens and their hidden state features in the question-answer pair response, the edges between nodes represent the dependencies between tokens, and the weight values of the edges are derived from the attention matrix.
[0009] In step S1, for a weighted directed graph: For a large language model with an L-layer transformer architecture, the input of the j-th layer comes from the hidden embeddings of the (j-1)-th layer, denoted as... , where t i Represents token i The hidden state, , After the multi-head self-attention mechanism of the j-th layer, the linear projection mapping is expressed as: in, , , For parameter matrices, Represents the query vector. and These represent the key vector and value vector, respectively; the attention matrix is obtained by measuring... and The similarity between them is calculated and used to capture the relationship between tokens. Its formal expression is as follows: Where, d k express Dimensions The scaling factor is used; the attention matrix is a lower triangular matrix, denoted as A. , Used to quantify from token i to token j Directed dependency relationship, This represents the attention value.
[0010] In step S2: A graph convolutional neural network is used to classify weighted directed graphs. The graph convolutional neural network adopts the GraphConv architecture and extracts global graph representation by iteratively aggregating node features and their neighbor information to achieve graph-level classification. A weighted directed graph G = (V, E, X, A), where V represents the set of nodes and E represents the set of edges. Represents the node feature matrix, Represent the adjacency matrix; if This indicates that there is no edge between nodes i and j. When graph G is processed by GNN, node i is at the _j_th ... l The feature matrix of the layer is represented as: Where D is a diagonal matrix, D ii and D jj W represents the in-degree of node i and node j, respectively. ji Let W represent the weight matrix from node j to node i, where W and b represent the trainable weights and bias parameters, respectively, and ReLU is the non-linear activation function. By stacking multiple layers of graph convolution, graph representations are extracted, and token-level features are aggregated into graph-level representations through global mean pooling. Where N is the number of nodes. Represent the global characteristics of a weighted directed graph; Input a fully connected neural network to obtain the probabilities of the real label and the hallucination label, expressed as: in, and This represents the weights and biases of the fully connected layer. It is a two-dimensional vector; The dataset is divided into training, validation and test sets. The training process on the training set follows the standard supervised training process, with the Adam optimizer and binary cross-entropy as the objective functions.
[0011] In step S3: Given a weighted directed graph G and its detection prediction Y, identify an interpretive subgraph. This subgraph is used to maximize Mutual information between Y and Y: in, express entropy, Based on subgraph Conditional entropy; due to As constants, maximizing mutual information is equivalent to minimizing conditional entropy: in, Indicates in the subgraph (Parameters are) Under these conditions, variables The probability of occurrence; transforming the discrete graph optimization problem into a continuous optimization scheme, introducing learnable mask parameters for nodes and weighted edges, for each node. Define learnable scalar parameters The node importance score is obtained by normalizing it to the interval [0, 1] using the sigmoid function. in, This indicates that node v is included in the interpretation subgraph. The probability of it; The original node features are scaled using a node mask to obtain the masked node feature matrix. , This represents the original features of node v. Represents element-wise multiplication; The importance score of the edge is Construct the mask adjacency matrix Explain the subgraph From the mask feature matrix and mask adjacency matrix definition, New mask image It contains continuous values, and the edge weights are restricted to the interval [0, 1]; this is achieved by introducing node mask parameters. With edge mask parameters The optimization objective is redefined as a continuous optimization problem with respect to the parameters, and the expression for the objective function L is: Where C represents the number of categories, In the original diagram The predicted probability of category c under the given conditions. In the mask subgraph Predicted probability under given conditions and These are the regularization coefficients that control the sparsity of node masks and edge masks, respectively. This indicates L1 norm regularization.
[0012] A second aspect of this invention discloses a large language model-based hallucination detection system based on graph neural networks, the system comprising: The first processing unit is configured to: construct a labeled sample set containing question-answer pairs; and couple the hidden state of a specific layer of the large language model with the attention matrix to establish a weighted directed graph. In these question-and-answer pairs, the answers are labeled as fact or illusion; The second processing unit is configured to input a weighted directed graph into a graph neural network-based illusion detector for supervised learning and training. The third processing unit is configured to: deploy a hallucination detector that has completed supervised learning and training, perform real-time hallucination detection on question-answer pairs, and simultaneously provide token-level interpretability analysis.
[0013] For weighted directed graphs: Weighted directed graphs are used to represent the hidden state and attention matrix of a specific layer in a coupled large language model. The nodes in the graph represent the tokens and their hidden state features in the question-answer pair response, the edges between nodes represent the dependencies between tokens, and the weight values of the edges are derived from the attention matrix.
[0014] For weighted directed graphs: For a large language model with an L-layer transformer architecture, the input of the j-th layer comes from the hidden embeddings of the (j-1)-th layer, denoted as... , where t i Represents token i The hidden state, , After the multi-head self-attention mechanism of the j-th layer, the linear projection mapping is expressed as: in, , , For parameter matrices, Represents the query vector. and These represent the key vector and value vector, respectively; the attention matrix is obtained by measuring... and The similarity between them is calculated and used to capture the relationship between tokens. Its formal expression is as follows: Where, d k express Dimensions The scaling factor is used; the attention matrix is a lower triangular matrix, denoted as A. , Used to quantify from token i to token j Directed dependency relationship, This represents the attention value.
[0015] The second processing unit is specifically configured as follows: A graph convolutional neural network is used to classify weighted directed graphs. The graph convolutional neural network adopts the GraphConv architecture and extracts global graph representation by iteratively aggregating node features and their neighbor information to achieve graph-level classification. A weighted directed graph G = (V, E, X, A), where V represents the set of nodes and E represents the set of edges. Represents the node feature matrix, Represent the adjacency matrix; if This indicates that there is no edge between nodes i and j. When graph G is processed by GNN, node i is at the _j_th ... l The feature matrix of the layer is represented as: Where D is a diagonal matrix, D ii and D jj W represents the in-degree of node i and node j, respectively. ji Let W represent the weight matrix from node j to node i, where W and b represent the trainable weights and bias parameters, respectively, and ReLU is the non-linear activation function. By stacking multiple layers of graph convolution, graph representations are extracted, and token-level features are aggregated into graph-level representations through global mean pooling. Where N is the number of nodes. Represent the global characteristics of a weighted directed graph; Input a fully connected neural network to obtain the probabilities of the real label and the hallucination label, expressed as: in, and This represents the weights and biases of the fully connected layer. It is a two-dimensional vector; The dataset is divided into training, validation and test sets. The training process on the training set follows the standard supervised training process, with the Adam optimizer and binary cross-entropy as the objective function.
[0016] The third processing unit is specifically configured as follows: Given a weighted directed graph G and its detection prediction Y, identify an interpretive subgraph. This subgraph is used to maximize Mutual information between Y and Y: in, express entropy, Based on subgraph Conditional entropy; due to As constants, maximizing mutual information is equivalent to minimizing conditional entropy: in, Indicates in the subgraph (Parameters are) Under these conditions, variables The probability of occurrence; transforming the discrete graph optimization problem into a continuous optimization scheme, introducing learnable mask parameters for nodes and weighted edges, for each node. Define learnable scalar parameters The node importance score is obtained by normalizing it to the interval [0, 1] using the sigmoid function. in, This indicates that node v is included in the interpretation subgraph. The probability of it; The original node features are scaled using a node mask to obtain the masked node feature matrix. , This represents the original features of node v. Represents element-wise multiplication; The importance score of the edge is Construct the mask adjacency matrix Explain the subgraph From the mask feature matrix and mask adjacency matrix definition, New mask image It contains continuous values, and the edge weights are restricted to the interval [0, 1]; this is achieved by introducing node mask parameters. With edge mask parameters The optimization objective is redefined as a continuous optimization problem with respect to the parameters, and the expression for the objective function L is: Where C represents the number of categories, In the original diagram The predicted probability of category c under the given conditions. In the mask subgraph Predicted probability under given conditions and These are the regularization coefficients that control the sparsity of node masks and edge masks, respectively. This indicates L1 norm regularization.
[0017] In summary, compared with other technical solutions, this invention achieves state-of-the-art (SOTA) performance; at the same time, due to the inherent advantages of graph structures, the importance of nodes and edges can be explicitly described, which provides an important basis for the interpretability of detection results. Attached Figure Description
[0018] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0019] Figure 1 This is a schematic diagram of a large language model illusion detection process based on graph neural networks provided in an embodiment of the present invention.
[0020] Figure 2 This is a schematic diagram of a weighted directed graph provided in an embodiment of the present invention.
[0021] Figure 3 This is an example of interpretability analysis of hallucination detection results based on a sample question-and-answer pair provided in an embodiment of the present invention. Detailed Implementation
[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0023] This invention proposes a large language model hallucination detection framework based on graph neural networks. The method comprises a three-stage process. The first stage constructs a labeled sample set containing a small number of question-answer pairs, where answers are labeled as fact or hallucination by experts. Then, the hidden state of a specific layer of the large language model is coupled with the attention matrix to represent it as a weighted directed graph. The second stage inputs the weighted directed graph into a graph neural network-based hallucination detector for supervised learning and training. The third stage deploys the hallucination detector to perform real-time hallucination detection on question-answer pairs and simultaneously provides token-level interpretability analysis.
[0024] This invention proposes a hallucination detection scheme based on graph neural networks for large language models. It can detect in real time whether the content generated by the large language model contains hallucinations, thus improving the reliability of the large language model. The detailed process is as follows: Figure 1 As shown.
[0025] 1. Construct a weighted directed graph First, a weighted directed graph is used to represent the hidden states and attention matrix coupled in the large language model. In this graph, nodes represent tokens in the response and their hidden state features, edges between nodes capture the dependencies between tokens, and edge weights are derived from the attention matrix. Specifically, for a large language model with an L-layer transformer architecture, the input to the j-th layer is the hidden embedding from the (j-1)-th layer, denoted as... ,in represent The hidden state, The feature dimensions vary across different large language model categories; for example, the Llama2 series uses a 4096-dimensional representation. Input After processing by the multi-head self-attention mechanism in the j-th layer, the linear projection mapping can be represented as: in , , All of these are trainable parameter matrices. Represents the query vector. and These represent the key vector and value vector, respectively. The attention matrix is calculated by measuring... and The similarity between tokens is calculated to capture the relationships between them, and its formal expression is shown in the following equation: Where, d k express Dimensions The scaling factor is used; the attention matrix is a lower triangular matrix, denoted as A. , Used to quantify from token i to token j Directed dependency relationship, This represents the attention value.
[0026] In summary, this method systematically extracts and couples hidden state and attention matrix through a weighted directed graph structure. To fully model the question-answering process, all context tokens from the query and response are incorporated into the graph structure. For example... Figure 2 As shown, edge weights are represented by color intensity—the darker the hue, the stronger the correlation between terms. Each graph is associated with a label (fact or illusion). These labeled graphs are fed as input to a graph neural network model for training, resulting in an illusion detection probe.
[0027] 2. Hallucination detector based on graph neural network This invention employs a graph convolutional neural network (GCNN) to classify weighted directed graphs, with the graph convolutional layer being the core component. The invention utilizes a general GraphConv architecture, iteratively aggregating node features and their neighbor information to progressively extract global graph representations for graph-level classification. Specifically, the following definition applies: Let the weighted directed graph G = (V, E, X, A), where V represents the set of nodes and E represents the set of edges. Represents the node feature matrix, This represents the adjacency matrix. If... If , it means there is no edge between nodes i and j. When graph G is processed by GNN, the feature matrix of node i in the l-th layer can be expressed as: Where D is a diagonal matrix. and Let i and j represent the in-degrees of node i and node j, respectively. Let W represent the weight matrix from node j to node i, where W and b represent the trainable weights and bias parameters, respectively, and ReLU is the non-linear activation function.
[0028] By stacking multiple layers of graph convolutions, a more comprehensive graph representation is gradually extracted. Finally, node-level (i.e., token-level) features are aggregated into a graph-level (i.e., response-level) representation through global mean pooling, as shown in the following equation: Where N is the number of nodes. This represents the global characteristics of the weighted directed graph. Then... The probability of obtaining the real label and the hallucination label is obtained by inputting a fully connected neural network, and its expression is: in and These represent the weights and biases of the fully connected layer, respectively. It is a two-dimensional vector, representing the membership degree of fact and illusion, respectively.
[0029] Finally, this method divides the dataset into training, validation, and test sets. The training process on the training set follows a standard supervised training process and is supplemented by the Adam optimizer and binary cross-entropy as the objective function.
[0030] 3. Real-time hallucination detection This stage deploys the trained hallucination detector in a real-world application to evaluate the authenticity of content generated by the large language model in real time. Existing white-box methods mainly utilize hidden state or attention matrix, making it extremely difficult to achieve interpretability of the detection results. The graph structure method proposed in this paper couples the hidden state with the attention matrix, resulting in better interpretability. This method is the first to conduct interpretability analysis of hallucination detection results from a token perspective, systematically exploring which tokens and their key relationships have a significant determining effect on the detection results.
[0031] Specifically, given a weighted directed graph G and its detection prediction result Y, the goal of this method is to identify an interpretive subgraph. This subgraph can maximize The mutual information between Y and Y is shown in the following formula: in express entropy, Based on subgraph The conditional entropy. Because As a constant, maximizing mutual information is equivalent to minimizing conditional entropy, as shown in the following equation: in Indicates in the subgraph (Parameters are) Under these conditions, variables The probability of occurrence. However, directly optimizing the subgraph... This leads to combinatorial optimization problems with computational complexity that is difficult to handle. To transform this discrete graph optimization problem into a continuous optimization scheme, this method introduces learnable mask parameters for nodes and weighted edges. Specifically, for each node... This method defines learnable scalar parameters. The node importance score is obtained by normalizing it to the interval [0, 1] using the sigmoid function. in, This indicates that node v is included in the interpretation subgraph. The probability of the nodes is then determined. Subsequently, this method uses a node mask to scale the original node features, obtaining a masked node feature matrix. .in, This represents the original features of node v. This represents element-wise multiplication. Similarly, this method derives the importance score for edges. And construct the mask adjacency matrix Explaining subgraphs From the mask feature matrix and mask adjacency matrix Formal definition, denoted as New mask image It contains continuous values, and the edge weights are restricted to the interval [0, 1]. This is achieved by introducing node mask parameters. With edge mask parameters This method redefines the optimization objective as a continuous optimization problem with respect to these parameters. The formal expression for the objective function L is: Where C represents the number of categories. In the original diagram The predicted probability of category c under the given conditions. In the mask subgraph Predicted probability under given conditions and These are the regularization coefficients that control the sparsity of node masks and edge masks, respectively. This indicates L1 norm regularization.
[0032] Based on the above interpretability analysis, this method can obtain visualizations based on hallucination detection results, and further analyze the impact of nodes and edges on the detection results, such as... Figure 3 As shown, this example demonstrates the interpretability analysis of hallucination detection results based on a sample question-and-answer pair. Our method finds that most tokens participate in the decision-making process for determining the authenticity of responses, but this is not the case for edges; only edges between a subset of nodes affect the prediction results.
[0033] In summary, this invention proposes a large language model hallucination detection method based on graph neural networks, which is the first work to utilize graph neural networks for hallucination detection. This method first couples the hidden state of the token and the attention matrix as a weighted directed graph, and then trains a simple yet effective graph convolutional neural network classifier. On the one hand, this method requires only a small amount of labeled sample data, demonstrating state-of-the-art performance on multiple question-answering datasets and different families of large language models. On the other hand, this method is the first to perform token-level interpretability analysis on white-box hallucination detection methods, overcoming the shortcomings of poor interpretability in existing methods, and providing a practical and efficient solution for the secure application of large language models in the real world.
[0034] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.
[0035] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.
[0036] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of the present invention, and will not be described in detail here.
[0037] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for hallucination detection based on a large language model using graph neural networks, characterized in that, The method includes: Step S1: Construct a labeled sample set containing question-answer pairs; and couple the hidden state of a specific layer of the large language model with the attention matrix to establish a weighted directed graph. In these question-and-answer pairs, the answers are labeled as fact or illusion; Step S2: Input the weighted directed graph into the illusion detector based on graph neural network for supervised learning and training; Step S3: Deploy the hallucination detector that has completed supervised learning and training, perform real-time hallucination detection on question-answer pairs, and simultaneously provide token-level interpretability analysis.
2. The hallucination detection method based on a large language model using graph neural networks according to claim 1, characterized in that, In step S1, a weighted directed graph is established, wherein: the weighted directed graph is used to represent the hidden state and attention matrix of a specific layer of the coupled large language model, the nodes in the graph represent the tokens and their hidden state features in the question-answer pair response, the edges between nodes represent the dependencies between tokens, and the weight values of the edges are derived from the attention matrix.
3. The hallucination detection method based on a large language model using graph neural networks according to claim 2, characterized in that, In step S1, for a weighted directed graph: For a large language model with an L-layer transformer architecture, the input of the j-th layer comes from the hidden embeddings of the (j-1)-th layer, denoted as... , where t i Represents token i The hidden state, , After the multi-head self-attention mechanism of the j-th layer, the linear projection mapping is expressed as: in, , , For parameter matrices, Represents the query vector. and These represent the key vector and value vector, respectively; the attention matrix is obtained by measuring... and The similarity between them is calculated and used to capture the relationship between tokens. Its formal expression is as follows: Where, d k express Dimensions The scaling factor is used; the attention matrix is a lower triangular matrix, denoted as A. , Used to quantify from token i to token j Directed dependency relationship, This represents the attention value.
4. The hallucination detection method based on a large language model using graph neural networks according to claim 3, characterized in that, In step S2: A graph convolutional neural network is used to classify weighted directed graphs. The graph convolutional neural network adopts the GraphConv architecture and extracts global graph representation by iteratively aggregating node features and their neighbor information to achieve graph-level classification. A weighted directed graph G = (V, E, X, A), where V represents the set of nodes and E represents the set of edges. Represents the node feature matrix, Represent the adjacency matrix; if This indicates that there is no edge between nodes i and j. When graph G is processed by GNN, node i is at the _j_th ... l The feature matrix of the layer is represented as: Where D is a diagonal matrix, D ii and D jj W represents the in-degree of node i and node j, respectively. ji Let W represent the weight matrix from node j to node i, where W and b represent the trainable weights and bias parameters, respectively, and ReLU is the non-linear activation function. By stacking multiple layers of graph convolution, graph representations are extracted, and token-level features are aggregated into graph-level representations through global mean pooling. Where N is the number of nodes. Represent the global characteristics of a weighted directed graph; Input a fully connected neural network to obtain the probabilities of the real label and the hallucination label, expressed as: in, and This represents the weights and biases of the fully connected layer. It is a two-dimensional vector; The dataset is divided into training, validation and test sets. The training process on the training set follows the standard supervised training process, with the Adam optimizer and binary cross-entropy as the objective functions.
5. The hallucination detection method based on a large language model using a graph neural network according to claim 4, characterized in that, In step S3: Given a weighted directed graph G and its detection prediction Y, identify an interpretive subgraph. This subgraph is used to maximize Mutual information between Y and Y: in, express entropy, Based on subgraph Conditional entropy; due to As constants, maximizing mutual information is equivalent to minimizing conditional entropy: in, Indicates in the subgraph (Parameters are) Under these conditions, variables The probability of occurrence; transforming the discrete graph optimization problem into a continuous optimization scheme, introducing learnable mask parameters for nodes and weighted edges, for each node. Define learnable scalar parameters The node importance score is obtained by normalizing it to the interval [0, 1] using the sigmoid function. in, This indicates that node v is included in the interpretation subgraph. The probability of it; The original node features are scaled using a node mask to obtain the masked node feature matrix. , This represents the original features of node v. Represents element-wise multiplication; The importance score of the edge is Construct the mask adjacency matrix Explain the subgraph From the mask feature matrix and mask adjacency matrix definition, New mask image It contains continuous values, and the edge weights are restricted to the interval [0, 1]; this is achieved by introducing node mask parameters. With edge mask parameters The optimization objective is redefined as a continuous optimization problem with respect to the parameters, and the expression for the objective function L is: Where C represents the number of categories, In the original diagram The predicted probability of category c under the given conditions. In the mask subgraph Predicted probability under given conditions and These are the regularization coefficients that control the sparsity of node masks and edge masks, respectively. This indicates L1 norm regularization.
6. A large language model-based hallucination detection system based on graph neural networks, characterized in that, The system includes: The first processing unit is configured to: construct a labeled sample set containing question-answer pairs; and couple the hidden state of a specific layer of the large language model with the attention matrix to establish a weighted directed graph. In these question-and-answer pairs, the answers are labeled as fact or illusion; The second processing unit is configured to input a weighted directed graph into a graph neural network-based illusion detector for supervised learning and training. The third processing unit is configured to: deploy a hallucination detector that has completed supervised learning and training, perform real-time hallucination detection on question-answer pairs, and simultaneously provide token-level interpretability analysis.
7. The hallucination detection system based on a large language model using a graph neural network according to claim 6, characterized in that, For weighted directed graphs: Weighted directed graphs are used to represent the hidden state and attention matrix of a specific layer in a coupled large language model. The nodes in the graph represent the tokens and their hidden state features in the question-answer pair response, the edges between nodes represent the dependencies between tokens, and the weight values of the edges are derived from the attention matrix.
8. The hallucination detection system based on a large language model using a graph neural network according to claim 7, characterized in that, For weighted directed graphs: For a large language model with an L-layer transformer architecture, the input of the j-th layer comes from the hidden embeddings of the (j-1)-th layer, denoted as... , where t i Represents token i The hidden state, , After the multi-head self-attention mechanism of the j-th layer, the linear projection mapping is expressed as: in, , , For parameter matrices, Represents the query vector. and These represent the key vector and value vector, respectively; the attention matrix is obtained by measuring... and The similarity between them is calculated and used to capture the relationship between tokens. Its formal expression is as follows: Where, d k express Dimensions The scaling factor is used; the attention matrix is a lower triangular matrix, denoted as A. , Used to quantify from token i to token j Directed dependency relationship, This represents the attention value.
9. A large language model hallucination detection system based on graph neural networks according to claim 8, characterized in that, The second processing unit is specifically configured as follows: A graph convolutional neural network is used to classify weighted directed graphs. The graph convolutional neural network adopts the GraphConv architecture and extracts global graph representation by iteratively aggregating node features and their neighbor information to achieve graph-level classification. A weighted directed graph G = (V, E, X, A), where V represents the set of nodes and E represents the set of edges. Represents the node feature matrix, Represent the adjacency matrix; if This indicates that there is no edge between nodes i and j. When graph G is processed by GNN, node i is at the _j_th ... l The feature matrix of the layer is represented as: Where D is a diagonal matrix, D ii and D jj W represents the in-degree of node i and node j, respectively. ji Let W represent the weight matrix from node j to node i, where W and b represent the trainable weights and bias parameters, respectively, and ReLU is the non-linear activation function. By stacking multiple layers of graph convolution, graph representations are extracted, and token-level features are aggregated into graph-level representations through global mean pooling. Where N is the number of nodes. Represent the global characteristics of a weighted directed graph; Input a fully connected neural network to obtain the probabilities of the real label and the hallucination label, expressed as: in, and This represents the weights and biases of the fully connected layer. It is a two-dimensional vector; The dataset is divided into training, validation and test sets. The training process on the training set follows the standard supervised training process, with the Adam optimizer and binary cross-entropy as the objective function.
10. A large language model hallucination detection system based on graph neural networks according to claim 9, characterized in that, The third processing unit is specifically configured as follows: Given a weighted directed graph G and its detection prediction Y, identify an interpretive subgraph. This subgraph is used to maximize Mutual information between Y and Y: in, express entropy, Based on subgraph Conditional entropy; due to As constants, maximizing mutual information is equivalent to minimizing conditional entropy: in, Indicates in the subgraph (Parameters are) Under these conditions, variables The probability of occurrence; transforming the discrete graph optimization problem into a continuous optimization scheme, introducing learnable mask parameters for nodes and weighted edges, for each node. Define learnable scalar parameters The node importance score is obtained by normalizing it to the interval [0, 1] using the sigmoid function. in, This indicates that node v is included in the interpretation subgraph. The probability of it; The original node features are scaled using a node mask to obtain the masked node feature matrix. , This represents the original features of node v. Represents element-wise multiplication; The importance score of the edge is Construct the mask adjacency matrix Explain the subgraph From the mask feature matrix and mask adjacency matrix definition, New mask image It contains continuous values, and the edge weights are restricted to the interval [0, 1]; this is achieved by introducing node mask parameters. With edge mask parameters The optimization objective is redefined as a continuous optimization problem with respect to the parameters, and the expression for the objective function L is: Where C represents the number of categories, In the original diagram The predicted probability of category c under the given conditions. In the mask subgraph Predicted probability under given conditions and These are the regularization coefficients that control the sparsity of node masks and edge masks, respectively. This indicates L1 norm regularization.