Hallucination detection system, hallucination detection method, and electronic device
Patent Information
- Authority / Receiving Office
- TW · TW
- Patent Type
- Patents
- Current Assignee / Owner
- INSTITUTE FOR INFORMATION INDUSTRY
- Filing Date
- 2024-10-17
- Publication Date
- 2026-08-01
AI Technical Summary
Existing large-scale language models generate answers prone to incorrect information, making it difficult for users to determine the accuracy of responses, especially when questions are outside their expertise, leading to user inconvenience.
A hallucination detection system using a pre-trained graph embedding model to generate authenticity information for output text, identifying real or false word networks and providing hallucination detection results, including the degree and specific words, by training on forward and backward word networks within a single knowledge domain.
Enables users to accurately identify and quantify hallucinations in large language model outputs, reducing the likelihood of accepting erroneous content and allowing developers to evaluate response reliability.
Smart Images
Figure TWG2TB001903651_001 
Figure TWG2TB001903651_002 
Figure TWG2TB001903651_003
Abstract
Description
Technical Field
[0001] This invention relates to a hallucination detection system, a hallucination detection method, and an electronic device, particularly a hallucination detection system, a hallucination detection method, and an electronic device including a hallucination detection system for detecting output text generated by a large language model. Prior Technology
[0002] Existing large-scale language models (LLMs) are trained using vast amounts of words and text. Therefore, the answers generated by these models based on user input are prone to providing incorrect information. When users ask questions outside their area of expertise, they may find it difficult to determine whether the answer contains incorrect information (a phenomenon known in the industry as "illusion"). This causes inconvenience for users. Summary of the Invention
[0003] This invention discloses a hallucination detection system, a hallucination detection method, and an electronic device, mainly used to improve the existing technology, which makes it difficult to perform hallucination detection on the output text generated by large language models.
[0004] One embodiment of the present invention discloses a hallucination detection system, comprising: a storage device and a processor. The storage device is used to store the output text of a large language model. The processor is coupled to the storage device. After receiving the output text, the processor can execute a hallucination detection method, which includes the following steps: a conversion step: generating multiple output word networks (Word Nets) based on the output text; a authenticity detection step: generating multiple authenticity information entries using a pre-trained graph embedding model; each authenticity information entry represents whether one of the output word networks is real or false after being judged by the graph embedding model; a result output step: generating hallucination detection result information based on the multiple authenticity information entries; the hallucination detection result information indicates whether the output text has hallucinations, the degree of hallucination in the output text, and at least one of the hallucinatory words in the output text; wherein, the pre-trained graph embedding model is trained using multiple forward word networks and multiple backward word networks in a single knowledge domain.
[0005] One embodiment of the present invention discloses an electronic device comprising the hallucination detection system and display of the present invention, wherein a processor is electrically connected to the display and the processor is capable of controlling the display to show hallucination detection result information.
[0006] One embodiment of the present invention discloses a hallucination detection method, which is used to perform hallucination detection on the output text of a large language model and generate hallucination detection result information. The hallucination detection method can be executed by a processor and includes the following steps: a conversion step: generating multiple output word networks (Word Nets) based on the output text; a authenticity detection step: generating multiple authenticity information using a pre-trained graph embedding model; each authenticity information represents whether one of the output word networks is real or false after being judged by the graph embedding model; a result output step: generating hallucination detection result information based on the multiple authenticity information; the hallucination detection result information indicates whether the output text has hallucinations, the degree of hallucination in the output text, and at least one of the hallucination words in the output text; wherein, the pre-trained graph embedding model is trained using multiple forward word networks and multiple backward word networks in a single knowledge domain.
[0007] In summary, the hallucination detection system, hallucination detection method, and electronic device of the present invention, through the design of the conversion step, the authenticity detection step, and the result output step, allow users to know whether the output text generated by the large language model has hallucination, the degree of hallucination, and at least one of the hallucination words in the output text by viewing the hallucination detection result information.
[0008] To further understand the features and technical content of this invention, please refer to the following detailed description and accompanying drawings. However, these descriptions and drawings are only for illustrating the invention and are not intended to limit the scope of protection of the invention in any way. Simple Explanation of the Diagram
[0009] Figure 1 is a block diagram of the hallucination detection system of the present invention.
[0010] Figure 2 is a schematic flowchart of the hallucination detection method of the present invention.
[0011] Figure 3 is a schematic diagram of the electronic device of the present invention. Implementation
[0012] In the following description, if it is indicated that a specific diagram is referred to or as shown in a specific diagram, it is only to emphasize that most of the relevant content mentioned in the following description appears in that specific diagram, but does not limit the following description to refer only to that specific diagram.
[0013] Please refer to Figures 1 to 3. Figure 1 is a block diagram of the hallucination detection system of the present invention. Figure 2 is a schematic diagram of the electronic device of the present invention. Figure 3 is a flowchart of the hallucination detection method of the present invention.
[0014] The hallucination detection system 100 of the present invention includes a storage device 1 and a processor 2. The hallucination detection system 100 of the present invention can be built into the electronic device 200 of the present invention, such as a server, computer (e.g., home computer, industrial computer, etc.), or portable electronic product (e.g., laptop, smartphone, tablet computer, etc.). In practical applications, the electronic device 200 may, for example, have a built-in large language model (LLM) and its application program.
[0015] Storage 1 is used to store the output text A1 of a large language model A. Storage 1 is primarily used to store the output text A1; therefore, in practice, the type and form of storage 1 can be selected according to actual needs and are not limited thereto. For example, storage 1 can be various types of memory, hard drives, etc. In different embodiments, storage 1 and processor 2 can be integrated into a processing module, with storage 1 serving as a memory unit within the processing module.
[0016] Processor 2 is coupled to memory 1. Processor 2 can obtain the output text A1 generated by the large language model A and store the output text A1 in memory 1. In practice, 2 can, for example, use an application programming interface (API) to obtain the output text A1 of the large language model A and store the output text A1 in memory 1.
[0017] After receiving the output text A1, processor 2 can execute the hallucination detection method of the present invention, which includes the following steps.
[0018] Conversion step S11: Generate a multi-word network based on the output text A1.
[0019] Authenticity detection step S12: Using a pre-trained graph embedding model, generate multiple pieces of true and false information; each piece of true and false information represents whether the output word network is real or false after being judged by the graph embedding model.
[0020] Result output step S13: Based on multiple pieces of true and false information, generate hallucination detection result information 21; Hallucination detection result information 21 indicates at least one of the following: whether the output text A1 has hallucination, the degree of hallucination of the output text A1, and the hallucinatory words contained in the output text A1.
[0021] The pre-trained graph embedding model is trained using multiple forward and backward word networks within a single knowledge domain. The scope of the single knowledge domain can be selected based on actual needs. For example, the scope of the single knowledge domain can be the legal domain, the labor law domain, etc., without limitation.
[0022] In practice, before training the graph embedding model, true and false labels can be added to each forward word network and each backward word network. Then, the graph embedding model is trained using the Graphic Neural Network (GNN) technique. In this way, the graph embedding model can learn the language features of each node in the word network, as well as the network structure features between multiple nodes, and can learn which feature combinations are true and which feature combinations are false.
[0023] As described above, the hallucination detection system of the present invention first converts the output text A1 of a large language model A into a network of multiple output words, and then uses a pre-trained graph embedding model to judge whether each output word network is real or false, thereby generating hallucination detection result information 21. This allows users to know whether the output text A1 generated by the large language model A is a hallucination (i.e., whether there are false words or false sentences in the output text A1) in a specific single knowledge domain (i.e., the single knowledge domain used to train the graph embedding model). In this way, it can reduce the possibility that users may mistakenly believe the erroneous content output by the large language model A, or provide the developers of the large language model A with the opportunity to evaluate whether the response content of this model is prone to hallucination.
[0024] In one embodiment, during the authenticity detection step S12, if the processor 2 determines that any output word network is false based on the graph embedding model, the illusion detection result information 21 generated by the processor 2 in the result output step S13 will indicate that the output text A1 is an illusion. For example, as shown in Figure 3, assuming that the electronic device 200 executes an application 202 similar to ChatGPT, the user inputs the question "How long can a pregnant woman take maternity leave?", and the output text generated by the large language model is "According to Article 50 of Chapter 5 of the Labor Standards Act, maternity leave is in principle eight weeks. However, maternity leave can only be granted with the employer's consent." After the processor 2 completes the above conversion step S11, authenticity detection step S12, and result output step S13, the processor 2 will control the display 201 to display the illusion detection result information 21, and the user can see text such as "illusion may exist" in the interface of the application 202.
[0025] The degree of hallucination can be generated based on the total number of true and false entries. The total number of true entries refers to the total number of output word networks that are determined to be true by the graph embedding model; the total number of false entries refers to the total number of output word networks that are determined to be false by the graph embedding model A1. For example, suppose the output text A1 is converted into 10 output word networks, and in the truth / false detection step S12, the processor 2 determines that 4 of the output word networks are true and the remaining 6 are false. Then, in the result output step S13, the hallucination detection result information 21 generated by the processor 2 contains a hallucination degree of 60%. As shown in Figure 3, the hallucination detection result information 21 seen by the user on the display 201 of the electronic device 200 may contain text such as "Hallucination degree: 60%".
[0026] In one embodiment, the hallucination detection result information 21 includes hallucination words, which are the root node words of the output word network in the output text A1 that are determined to be false by the graph embedding model A. In another embodiment, the hallucination detection result information 21 may be a sentence containing hallucination words. For example, suppose the graph embedding model is trained using a forward word network and a backward word network generated by the "Labour Standards Act," and the output text A1 of the large language model A is "According to Article 50 of Chapter 5 of the Labor Standards Act, maternity leave is in principle eight weeks. However, maternity leave can only be granted with the employer's consent." If the output word network judged as false by the graph embedding model is "maternity leave → must → employer → consent → can → leave," then the node words contained in the output word network, such as "maternity leave," will be judged as hallucinatory words. In the hallucination detection result information 21, it can be a prompt to the user that the sentence corresponding to the output word network (i.e., maternity leave can be granted with the employer's consent) and all the words contained in the output word network (i.e., maternity leave, must, employer, consent, can, leave) may be hallucinatory. As shown in Figure 3, in the electronic device 200, the user can see text such as "Possible hallucinations: Maternity leave can only be taken with the employer's consent" in the hallucination detection result information 21.
[0027] It should be noted that, as shown in Figure 3, the hallucination detection result information 21 simultaneously includes three detection results: whether hallucination exists, the degree of hallucination, and hallucination words. However, the hallucination detection result information 21 is not limited to including all three detection results at the same time. In different embodiments, the hallucination detection result information 21 may also include only one of the three detection results, or include two of the three detection results.
[0028] In practical applications, in the conversion step S11, the output text A1 can first be processed by at least one of the following: sentence segmentation, word segmentation, removal of words with preset parts of speech, and removal of preset symbols, to generate multiple output words with sequential order. Then, these multiple output words are input into a pre-trained language feature model to establish a multiple output word network. The method for establishing the output word network can be designed according to actual needs and is not limited here. For example, in one embodiment, one output word network can be established using each sentence of the output text A1. In another embodiment, multiple output word networks can be established using multiple words of the output text A1. In yet another embodiment, multiple output word networks can be established using multiple paragraphs of the output text A1. The methods described in the above three embodiments can also coexist in one embodiment. Any two of the above three embodiments can also be combined to form a new embodiment.
[0029] The purpose of removing words with preset parts of speech is primarily to remove meaningless words such as particles and interjections, while retaining words with meaningful parts of speech, such as subjects, verbs, nouns, and adjectives. The methods for determining the parts of speech of each word are well-known techniques and will not be elaborated upon here. The purpose of removing preset symbols is primarily to remove meaningless symbols, such as line breaks, spaces, and exclamation marks.
[0030] In practical applications, processor 2 can perform the following steps to train a language feature model: Domain word generation step: processing multiple domain knowledge texts by segmentation, word segmentation, removal of words with preset parts of speech, and removal of preset symbols to generate multiple domain words with sequential order; and model training step: inputting the multiple domain words with sequential order into the Word2Vec model to train the language feature model.
[0031] Among them, multiple domain knowledge texts refer to texts within a single knowledge domain. For example, if the single knowledge domain is the "Labour Standards Act", then the various domain knowledge texts could be different articles of the Labor Standards Act.
[0032] After training the language feature model through the above steps, the correspondence between words and vectors in a single knowledge domain can be obtained, and the correlation between specific words and other words in that single knowledge domain can be determined. For example, if the language feature model is trained using the "Labour Standard Method," then inputting "maternity leave" into the language feature model will yield words that are highly correlated with "maternity leave" in the "Labour Standard Method." Similarly, the language feature model can also be used to obtain words that have a low correlation with "maternity leave" in the "Labour Standard Method."
[0033] As described above, for example, in conversion step S11, the output text A1 can first be segmented using periods, and then each sentence can be segmented using techniques such as Jieba and spaCy. In practical applications, when segmenting each sentence, techniques such as Jieba and spaCy can be used, along with pre-stored multi-domain thesaurus and dictionaries, so that each sentence can be segmented more accurately.
[0034] By first segmenting the output text A1 into sentences and then further segmenting each sentence into words, the resulting multiple output words can be ordered. For example, if the output text is "According to Article 50 of Chapter 5 of the Labor Standards Act, maternity leave is in principle eight weeks. However, maternity leave can only be granted with the employer's consent.", after segmentation, we get two sentences: "According to Article 50 of Chapter 5 of the Labor Standards Act, maternity leave is in principle eight weeks" and "However, maternity leave can only be granted with the employer's consent." After segmentation of the first sentence, we get eight output words, such as "Labor Standards Act," "Chapter 5," "Article 50," "stipulate," "maternity leave," "in principle," "have," and "eight weeks." These eight output words come from the same sentence, so during program processing, the eight output words are ordered. They can be stored using formats that can express order, such as arrays or dictionaries.
[0035] As described above, by processing the output text through sentence segmentation and word segmentation, a multi-stroke output word network with sequential order can be generated. Furthermore, the language feature model can represent the vectors corresponding to each word in a single knowledge domain. Therefore, by inputting the multi-stroke output word network with sequential order into the language feature model, a multi-stroke output word network can be obtained.
[0036] Following the example above, after the first sentence is broken down into 8 output words, these 8 sequential output words are input into the trained language feature model to obtain a multi-word output network corresponding to the first sentence, such as: "Article 50 of Chapter 5 of the Labor Standards Act stipulates maternity leave", "Article 50 of Chapter 5 stipulates maternity leave in principle"... "Maternity leave is in principle eight weeks", etc.
[0037] As described above, by training a language feature model using multiple domain knowledge texts within a single knowledge domain, multiple forward word networks and multiple reverse word networks can be established within that domain. Specifically, before processor 2 executes the authenticity detection step S12, it first performs the following steps: a forward training data generation step: inputting multiple domain knowledge texts into the language feature model to generate multiple forward word networks; and a reverse training data generation step: using the language feature model, by replacing the root node words of each forward word network with low-relevance words, to generate multiple reverse word networks.
[0038] As explained earlier, since a language feature model trained using a single knowledge domain can represent the relationships between words in different domains within that domain, after establishing multiple forward word networks, the language feature model can be used to find low-relevance words with vector distances relatively large to the root nodes of each forward word network. Thus, multiple backward word networks can be established using these low-relevance words and the multiple forward word networks. Specifically, the language feature model projects each word onto a vector; therefore, cosine similarity can be used to calculate the distance between two vectors to determine the relevance between the two words.
[0039] It should be noted that the process of establishing the forward and reverse word networks can be completed automatically by the program without any human intervention, thus saving a lot of time and manpower costs.
[0040] If a knowledge graph is used to build forward and reverse training data for a language feature model, a significant amount of manpower and time is required in the initial stages of knowledge graph construction. This involves manually labeling the relationships between every word and phrase in the text and sentences, establishing the relationships between words in the knowledge graph. Therefore, the personnel responsible for labeling must possess knowledge in this technical field, especially a clear understanding of the true meaning of the text being labeled, in order to correctly label the relationships between each word and phrase. In other words, if a language feature model based on the Labor Standards Act is to be built using a knowledge graph to create forward and reverse training data, the relevant personnel must be familiar with all the articles and specific terms of the Labor Standards Act to correctly label the relationships between the words and phrases within each article of the Labor Standards Act.
[0041] For example, when manually annotating Article 50 of Chapter 5 of the Labor Standards Act, which states "Female workers should stop working before and after childbirth and be granted eight weeks of maternity leave," using a knowledge graph, the relevant personnel must label "Labor Standards Act" as "Law," and label "Chapter 5" and "Article 50" as "Subsidiary Regulation" and "Subsidiary Regulation," respectively. They must also label the relationships between "Labor Standards Act," "Chapter 5," and "Article 50" as follows: "Labor Standards Act" includes "Chapter 5," "Chapter 5" belongs to "Labor Standards Act," "Chapter 5" includes "Article 50," and "Article 50" belongs to "Chapter 5." Furthermore, they must label "Rights" as a protected item, "Female Workers" as the protected object, and the relationships between "Article 50," "Rights," and "Female Workers" as follows: "Article 50" includes "Rights" and "Female Workers," "Rights" belongs to "Article 50," and "Female Workers" belongs to "Article 50."
[0042] In summary, the hallucination detection system, hallucination detection method, and electronic device of the present invention, through the design of conversion steps, authenticity detection steps, and result output steps, allow users to know whether the output text generated by the large language model has hallucination, the degree of hallucination, and at least one of the hallucination words in the output text by viewing the hallucination detection result information.
[0043] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Therefore, all equivalent technical changes made based on the description and drawings of the present invention are included within the protection scope of the present invention.
[0044] 100: Hallucination Detection System 200: Electronic devices 201: Monitor 202: Application 1: Storage 2: Processor 21: Hallucination Test Results Information A: Large-scale language model A1: Output text S11: Conversion Steps S12: Authenticity Verification Steps S13: Result Output Steps
Claims
1. A hallucination detection system, comprising: a storage device for storing an output text of a large language model; and a processor coupled to the storage device, wherein the processor, upon receiving the output text, is capable of executing a hallucination detection method, comprising the following steps: a conversion step: generating multiple output word networks based on the output text; a authenticity detection step: generating multiple pieces of authenticity information using a pre-trained graph embedding model; each piece of authenticity information represents whether one of the output word networks is real or false after being judged by the graph embedding model; and a result output step: generating hallucination detection result information based on the multiple pieces of authenticity information; the hallucination detection result information indicates whether the output text is hallucinatory, the degree of hallucination in the output text, and at least one of a hallucinatory word in the output text; the degree of hallucination is generated based on a total number of real words and a total number of false words; wherein, The pre-trained graph embedding model is trained using a multi-stroke forward word network and a multi-stroke backward word network within a single knowledge domain.
2. The hallucination detection system as described in claim 1, wherein, In the authenticity detection step, if the processor determines that any of the output word networks is false based on the graph embedding model, the illusion detection result information generated by the processor in the result output step will indicate that the output text is an illusion.
3. The hallucination detection system as described in claim 1, wherein, The total number of true words refers to the total number of output word networks in the output text that are determined to be true by the graph embedding model; the total number of false words refers to the total number of output word networks in the output text that are determined to be false by the graph embedding model.
4. The hallucination detection system as described in claim 1, wherein, The illusionary word is a node word in the output text that is determined to be false by the graph embedding model.
5. The hallucination detection system as described in claim 1, wherein, Before the processor executes the authenticity detection step, it first performs the following steps: A positive training data generation step: inputting multiple domain knowledge texts into a language feature model to generate multiple positive word networks; A reverse training data generation step: Using the language feature model, by replacing a node word in each of the forward word networks with a low-relevance word, multiple reverse word networks are generated.
6. The hallucination detection system as described in claim 5, wherein, Before the positive training data generation step, the following steps are also included: A domain word generation step: performing at least one of the following processing on multiple domain knowledge texts: sentence segmentation, word segmentation, removal of words with preset parts of speech, and removal of preset symbols, to generate multiple domain words; And a model training step: input multiple domain words into the Word2Vec model to train the language feature model.
7. The hallucination detection system as described in claim 5, wherein, In the conversion step, the output text is first processed by at least one of the following: sentence segmentation, word segmentation, removal of words with preset parts of speech, and removal of preset symbols, to generate multiple output words. Then, the multiple output words are input into the language feature model to establish a network of multiple output words.
8. An electronic device comprising a hallucination detection system as described in any one of claims 1 to 7 and a display, wherein the processor is electrically connected to the display and the processor is capable of controlling the display to display the hallucination detection result information.
9. A hallucination detection method for performing hallucination detection on an output text of a large language model and generating hallucination detection result information; the hallucination detection method can be executed by a processor, and the hallucination detection method includes the following steps: a conversion step: generating multiple output word networks (Word Nets) based on the output text; a authenticity detection step: generating multiple authenticity information entries using a pre-trained graph embedding model; each authenticity information entry represents whether one of the output word networks is real or false after being judged by the graph embedding model; and a result output step: generating hallucination detection result information based on the multiple authenticity information entries; the hallucination detection result information indicates whether the output text has a hallucination, the degree of hallucination of the output text, and at least one of a hallucination word in the output text; the degree of hallucination is generated based on a total number of real and a total number of false; wherein, The pre-trained graph embedding model is trained using a multi-stroke forward word network and a multi-stroke backward word network within a single knowledge domain.
10. The hallucination detection method as described in claim 9, wherein, In the authenticity detection step, if the processor determines that any of the output word networks is false based on the graph embedding model, the illusion detection result information generated by the processor in the result output step will indicate that the output text is an illusion.
11. The hallucination detection method as described in claim 9, wherein, The total number of true words refers to the total number of output word networks in the output text that are determined to be true by the graph embedding model; the total number of false words refers to the total number of output word networks in the output text that are determined to be false by the graph embedding model.
12. The hallucination detection method as described in claim 9, wherein, The illusionary word is a node word in the output word network that is determined to be false by the graph embedding model in the output text.
13. The hallucination detection method as described in claim 9, wherein, Before the authenticity detection step, the following steps are also included: a positive training data generation step: inputting multiple domain knowledge texts into a language feature model to generate multiple positive word networks; A reverse training data generation step: Using the language feature model, by replacing a node word in each of the forward word networks with a low-relevance word, multiple reverse word networks are generated.
14. The hallucination detection method as described in claim 13, wherein, Before the positive training data generation step, the following steps are also included: A domain word generation step: performing at least one of the following processing on multiple domain knowledge texts: sentence segmentation, word segmentation, removal of words with preset parts of speech, and removal of preset symbols, to generate multiple domain words; And a model training step: input multiple domain words into the Word2Vec model to train the language feature model.
15. The hallucination detection method as described in claim 13, wherein, In the conversion step, the output text is first processed by at least one of the following: sentence segmentation, word segmentation, removal of words with preset parts of speech, and removal of preset symbols, to generate multiple output words. Then, the multiple output words are input into the language feature model to establish a network of multiple output words.