Hallucination determination program, method, and device

Abstract

In the present invention, a first word included in a first sentence or an attribute of the first sentence is extracted on the basis of a first sentence outputted from a prescribed model, a second sentence is generated which is a query inputted to the prescribed model corresponding to the first sentence and has an embedded input section generated according to an attribute, and on the basis of a plurality of answers to the second sentence outputted by the prescribed model, it is determined whether the first sentence was a hallucination, whereby hallucinations are detected with high accuracy.

Description

Hallucination detection program, method, and device

[0001] The present invention relates to a program, a method and an apparatus.

[0002] 2. Description of the Related Art An interactive generative AI (Artificial Intelligence) system that uses a large language model (LLM) is known.

[0003] LLM is a deep learning model that uses neural networks with hundreds of billions of parameters to be used as language models, and is particularly specialized for predicting the next word. Interactive generative AI uses LLM to continuously predict the next word in response to a question posed by a user, and then outputs the generated answer.

[0004] LLM is prone to hallucinations due to the nature of outputting answers by continually predicting the next word. Hallucinations are plausible lies with no basis, and are the phenomenon of inventing stories that were not taught during training. These hallucinations make it difficult to apply AI to business applications.

[0005] Recently, a self-checking hallucination detection method has been proposed, which uses the interactive generative AI itself to detect hallucinatory sentences from sentences written by the interactive generative AI.

[0006] For example, if the same question is repeatedly asked to the interactive generation AI, and the more contradictory answers the interactive generation AI returns each time the question is reasked, the stronger the suspicion of hallucination becomes.On the other hand, if the interactive generation AI returns a supporting answer even when the question is reasked, the stronger the denial of hallucination becomes.

[0007] Ayush Agrawal, Mirac Suzgun, Lester Mackey, Adam Tauman Kalai, “Do Language Models Know When They're Hallucinating References?” [online], May 29, 2023, arXiv, [Retrieved October 13, 2023], Internet <URL: arxiv.org / abs / 2305.18248> Potsawee Manakul, Adian Liusie, Mark J.F. Gales, “SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models”, [online], March 15, 2023, Semantic Scholar, [Retrieved October 13, 2020], Internet <URL: / www.semanticscholar.org / paper / SelfCheckGPT%3A-Zero-Resource-Black-Box-Hallucination-Manakul-Liusie / 7c1707db9aafd209aa93db3251e7ebd593d55876> Miaoran Li, Baolin Peng, Zhu Zhang, “Self-Checker: Plug-and-Play Modules for Fact-Checking with Large Language Models”, [online], May 24, 2023, Semantic Scholar, [Retrieved October 13, 2023], Internet <URL: www.semanticscholar.org / paper / Self-Checker%3A-Plug-and-Play-Modules-for-with-Large-Li-Peng / 85b5068d3e1364b44ec9f46b0930b521b4089df6>

[0008] However, conventional hallucination detection methods have the problem that they are not accurate enough in detecting hallucinations from the output of interactive generation AI.

[0009] For example, because the method is specialized for bibliographic information (titles and author names), it may be difficult to apply it to domains other than bibliographic information. Here, a domain refers to the type of text content, such as sports news, historical novels, or life science papers.

[0010] Furthermore, when the same question is repeatedly asked of the interactive generation AI, multiple outputs (answer sentences) are obtained, and particularly when the output is complex, some of the multiple output answer sentences may not contain the words used to determine whether or not the answer is a hallucination. This can make it difficult to properly compare the multiple answer sentences, making it unclear whether they are true or false and making it impossible to properly verify them.

[0011] In one aspect, the present invention aims to enable hallucinations to be detected with high accuracy.

[0012] Therefore, this program causes a computer to execute the following process: based on a first sentence output from a predetermined model, extract a first word contained in the first sentence or an attribute of the first sentence; generate a second sentence that is a query to be input to the predetermined model corresponding to the first sentence, the second sentence having an input portion generated according to the attribute embedded therein; and determine whether the first sentence is a hallucination based on multiple answers to the second sentence output by the predetermined model.

[0013] According to one embodiment, hallucinations can be detected with high accuracy.

[0014] 1 is a diagram illustrating the functional configuration of the hallucination detection device according to the first embodiment. FIG. 2 is a diagram schematically illustrating the hardware configuration of a computer that realizes the functions of the hallucination detection device shown in FIG. 1. FIG. 3 is a diagram illustrating the data structure of LLM information in the hallucination detection device according to the first embodiment. FIG. 4 is a diagram illustrating the data structure of dialogue history information in the hallucination detection device according to the first embodiment. FIG. 5 is a diagram illustrating the data structure of fill-in history information in the hallucination detection device according to the first embodiment. FIG. 6 is a diagram illustrating a first semantic analysis method by a semantic analysis unit of the hallucination detection device according to the first embodiment. FIG. 7 is a diagram illustrating a second semantic analysis method by a semantic analysis unit of the hallucination detection device according to the first embodiment. FIG. 8 is a diagram illustrating the processing of a fill-in-the-blank question generation unit of the hallucination detection device according to the first embodiment. FIG. 9 is a diagram illustrating the processing of registering fill-in-the-blank questions generated by the fill-in-the-blank question generation unit in fill-in history information in the hallucination detection device according to the first embodiment. FIG. 10 is a diagram illustrating the processing of a fill-in-the-blank answer unit of the hallucination detection device according to the first embodiment. FIG. 11 is a diagram illustrating the processing of a hallucination presence / absence determination unit of the hallucination detection device according to the first embodiment. FIG. 12 is a flowchart illustrating the processing of the hallucination detection device according to the first embodiment. 1 is a diagram for explaining a false positive case in which a non-hallucinatory object is mistaken for a hallucination. FIG. 1 illustrates an example of the data structure of dialogue history information in the hallucination detection device according to the second embodiment. FIG. 2 illustrates an example of the data structure of fill-in-the-blank history information in the hallucination detection device according to the second embodiment. FIG. 3 is a diagram for explaining a first semantic analysis method by the semantic analysis unit of the hallucination detection device according to the second embodiment. FIG. 4 is a diagram for explaining a second semantic analysis method by the semantic analysis unit of the hallucination detection device according to the second embodiment. FIG. 5 is a diagram for explaining the processing of a fill-in-the-blank question generation unit of the hallucination detection device according to the second embodiment. FIG. 6 is a diagram for explaining the processing of a fill-in-the-blank question generated by the fill-in-the-blank question generation unit in the hallucination detection device according to the second embodiment, to register the fill-in-the-blank questions generated by the fill-in-the-blank question generation unit in the fill-in-the-blank history information. FIG. 7 is a diagram for explaining the processing of a fill-in-the-blank answer unit of the hallucination detection device according to the second embodiment. FIG. 8 is a diagram for explaining the processing of a hallucination presence / absence determination unit of the hallucination detection device according to the second embodiment. FIG. 9 is a flowchart for explaining the processing of the hallucination detection device according to the second embodiment.FIG. 10 is a diagram showing a modified example of a fill-in-the-blank problem in the hallucination detection device according to the first embodiment.

[0015] Hereinafter, embodiments of the present program, method, and apparatus will be described with reference to the drawings. However, the embodiments shown below are merely examples, and are not intended to exclude the application of various modifications and techniques not explicitly stated in the embodiments. In other words, the present embodiments can be implemented in various ways (such as by combining the embodiments and modifications) without departing from the spirit of the present invention. Furthermore, each figure does not intend to include only the components shown in the figure, but may include other functions, etc.

[0016] (I) Description of the First Embodiment FIG. 1 is a diagram illustrating the functional configuration of a hallucination detection device 1 according to the first embodiment, and FIG. 2 is a diagram schematically illustrating the hardware configuration of a computer 10 that realizes the functions of the hallucination detection device 1 shown in FIG. 1.

[0017] The hallucination detection device 1 realizes a function (hallucination detection function) of detecting hallucinations from the output (answer) of the interactive generation AI. The hallucination detection device 1 is an example of a device. The interactive generation AI outputs an answer generated by continuously predicting the next word using an LLM in response to a question given by a user (user question). Hereinafter, the answer output by the interactive generation AI is referred to as the AI ​​answer. The AI ​​answer is an example of a first sentence output from the LLM (predetermined model).

[0018] The hallucination detection device 1 receives a user question and an AI answer and determines whether or not a hallucination occurs in response to the AI ​​answer. For convenience, the interactive generation AI and the LLM may be treated on the same level. The LLM is an example of a predetermined model.

[0019] When multiple computers are used as hardware resources for realizing the functions of the hallucination detection device 1, each computer may have the hardware configuration illustrated in FIG. 2.

[0020] (A) Example of Hardware Configuration As shown in FIG. 2, the computer 10 may, for example, have a hardware configuration including a processor 11, a main memory 12, a storage device 13, an input / output interface (I / F) 14, an input / output device 15, a reading device 16, and a network interface (I / F) 17.

[0021] The processor 11 is an example of a processing unit that performs various controls and calculations, and is an example of a control unit. The processor 11 may be connected to each block in the computer 10 via a bus 18 so that they can communicate with each other. The processor 11 may be a multiprocessor including multiple processors, a multi-core processor having multiple processor cores, or a configuration having multiple multi-core processors.

[0022] Examples of the processor 11 include integrated circuits (ICs) such as a CPU, MPU, APU, DSP, ASIC, and FPGA. Note that the processor 11 may be a combination of two or more of these integrated circuits. CPU is an abbreviation for Central Processing Unit, MPU is an abbreviation for Micro Processing Unit, APU is an abbreviation for Accelerated Processing Unit, DSP is an abbreviation for Digital Signal Processor, ASIC is an abbreviation for Application Specific IC, and FPGA is an abbreviation for Field-Programmable Gate Array.

[0023] The input / output interface 14 controls, for example, the screen display of an output device such as a monitor in the input / output device 15. The input / output interface 14 may be a graphics processing device.

[0024] The input / output interface 14 may be configured as an accelerator that executes machine learning processing and inference processing using a machine learning model. The accelerator may be realized by various types of processing devices, such as a graphics processing unit (GPU), an APU, a DSP, an ASIC, an FPGA, or other integrated circuits (ICs).

[0025] The input / output device 15 may include one or both of an input device and an output device. Examples of input devices include a keyboard, a mouse, and a touch panel. Examples of output devices include a monitor, a projector, and a printer. The input / output device 15 may also include a touch panel or the like that combines an input device and a display device. The output device may be connected to the input / output interface 14.

[0026] User questions and AI responses, which will be described later, may be input to the hallucination detection device 1 via the input / output device 15. In addition, a statement explaining whether or not a hallucination is present, which will be described later, may be output via the input / output device 15.

[0027] The main memory 12 is an example of hardware that stores various types of data, programs, and other information. The main memory 12 may be, for example, a volatile memory such as a dynamic random access memory (DRAM) or a non-volatile memory such as a persistent memory (PM), or both.

[0028] The storage device 13 is an example of hardware that stores various types of data, programs, and other information. Examples of the storage device 13 include various storage devices such as magnetic disk devices such as HDDs (Hard Disk Drives), semiconductor drive devices such as SSDs (Solid State Drives), and non-volatile memories. Examples of non-volatile memories include flash memories, SCMs (Storage Class Memory), and ROMs (Read Only Memory).

[0029] The storage device 13 may store a program 34 that realizes all or part of the various functions of the hallucination detection device 1.

[0030] For example, the processor 11 can implement the hallucination detection function described below by expanding the program 34 stored in the storage device 13 into the main memory 12 and executing it.

[0031] The network interface 17 is an example of a communication IF that controls the connection and communication between the computer 10 and other computers. For example, the network interface 17 may include an adapter that complies with a LAN (Local Area Network) such as Ethernet (registered trademark) or optical communication such as FC (Fibre Channel). The adapter may support either or both wireless and wired communication methods.

[0032] For example, the computer 10 communicates with the LLM's Web API (Application Programming Interface) via the network interface 17 and a network (not shown).

[0033] The program 34 may be downloaded from the network to the computer 10 via the network interface 17 and stored in the storage device 13 .

[0034] The reading device 16 is an example of a reader that reads data and program information recorded on the recording medium 16a. The reading device 16 may include a connection terminal or device to which the recording medium 16a can be connected or inserted. Examples of the reading device 16 include an adapter that complies with USB (Universal Serial Bus) or the like, a drive device that accesses a recording disk, and a card reader that accesses a flash memory such as an SD card. Note that the recording medium 16a may store a program 34, and the reading device 16 may read the program 34 from the recording medium 16a and store it in the storage device 13.

[0035] Examples of the recording medium 16a include non-transitory computer-readable recording media such as magnetic / optical disks and flash memories. Examples of magnetic / optical disks include flexible disks, CDs (Compact Discs), DVDs (Digital Versatile Discs), Blu-ray Discs, and HVDs (Holographic Versatile Discs). Examples of flash memories include semiconductor memories such as USB memories and SD cards.

[0036] The above-described hardware configuration of the computer 10 is an example. Therefore, the hardware within the computer 10 may be increased or decreased (for example, any block may be added or removed), divided, or integrated in any combination, or buses may be added or removed, as appropriate.

[0037] 1, the hallucination detection device 1 may illustratively include functions as an LLM storage processor 21, a dialogue storage processor 22, a fill-in-the-blank storage processor 23, a fill-in-the-blank question generator 24, a fill-in-the-blank answer processor 25, a hallucination presence / absence determination unit 26, and a semantic analyzer 27. These functions may be realized by a processor 11 of a computer 10 (see FIG. 2).

[0038] The LLM storage processing unit 21 manages the LLM information 31 .

[0039] FIG. 3 is a diagram illustrating the data structure of the LLM information 31 in the hallucination detection device 1 according to the first embodiment.

[0040] The LLM information 31 is used to manage information (usage information) for using the LLM. The LLM information 31 may be, for example, a database. The LLM used by the hallucination detection device 1 may be different from or the same as the LLM used by the interactive generation AI that outputs the AI ​​answer.

[0041] The LLM information 31 shown in FIG. 3 manages information for one LLM in one row, and has a name column, a type column, and a model data column.

[0042] The name column records the name of the LLM. In the LLM information 31 illustrated in Fig. 3, mpt-7b may represent MPT-7B, which is an LLM of OSS (Open Source Software).

[0043] The type column records local or remote. The type of LLM that runs on the hallucination detection device 1 is local. The type of LLM that runs on another information processing device is remote. A remote LLM may be used via a Web API.

[0044] The model data string records LLM parameters. For local LLMs, the LLM parameters are recorded in a binary format. The binary format may be, for example, the PyTorch format.

[0045] For an LLM whose type is remote, connection information such as a uniform resource locator (URL) of the Web API is recorded in text format in the model data column.

[0046] The LLM Web API may be, for example, the Open Artificial Intelligence Chat Completion API, which is a Web API version of ChatGPT.

[0047] The LLM storage processing unit 21 acquires, for example, information about the LLM input by the user and records it in the LLM information 31 .

[0048] The dialogue storage processing unit 22 manages dialogue history information 32 .

[0049] FIG. 4 is a diagram illustrating an example of the data structure of the dialogue history information 32 in the hallucination detection device 1 according to the first embodiment.

[0050] The dialogue history information 32 is information that represents a dialogue history with the LLM. The dialogue history information 32 may be, for example, a database.

[0051] 4 manages one set of a user question and an AI answer in one row, and has a sequence number column, a user question column, and an AI answer column. A combination of a user question and an AI answer can be called a dialogue.

[0052] The user question and the AI ​​answer may be input to the hallucination detection device 1 via the input / output device 15 described above.

[0053] The dialogue storage processor 22 registers the dialogue history for which hallucination detection is desired in the dialogue history information 32. Identification information for identifying the dialogue is recorded in the serial number string. In the example shown in FIG. 4, integers are used as the identification information.

[0054] The user question column records questions (user questions) input to the LLM. The user questions may be created by the user of the hallucination detection device 1, or may be created by someone other than the user of the hallucination detection device 1. The AI ​​answer column records answers (AI answers) output by the LLM in response to the user questions. In this first embodiment, the AI ​​answers are assumed to be affirmative sentences.

[0055] In the example shown in FIG. 4, for example, a user question "Where is the headquarters of XX Company?" is associated with an AI answer "XX Company's headquarters is in Tokyo" and recorded.

[0056] The dialogue storage processing unit 22 acquires, for example, a user question and an AI response to the user question input via the input / output device 15 , and records them in the dialogue history information 32 .

[0057] The fill-in storage processing unit 23 manages fill-in history information 33 .

[0058] FIG. 5 is a diagram illustrating an example of the data structure of the filling history information 33 in the hallucination detection device 1 according to the first embodiment.

[0059] The fill-in-the-blank history information 33 is information that manages fill-in-the-blank questions created by the fill-in-the-blank question generation unit 24 (described later) and answers (fill-in-the-blank answers) to these fill-in-the-blank questions obtained from the LLM by the fill-in-the-blank answer unit 25. The fill-in-the-blank history information 33 may be, for example, a database.

[0060] The fill-in-the-blank history information 33 illustrated in FIG. 5 has an AI answer column, a fill-in-the-blank question column, a comparison column, and a fill-in-the-blank answer column, and each row manages the fill-in-the-blank question column, comparison column, and fill-in-the-blank answer column for one AI answer.

[0061] The fill-in-the-blank question column records fill-in-the-blank questions created by the fill-in-the-blank question generation unit 24. Details of the fill-in-the-blank questions will be described later.

[0062] In the comparison target column, the object (filled-in portion, input portion: details will be described later) left blank in the fill-in-the-blank question created by the fill-in-the-blank question generation unit 24 is recorded.

[0063] The fill-in-the-blank answer column records the answers output by the LLM for the fill-in-the-blank questions. As will be described later, the fill-in-the-blank answer unit 25 requests the LLM to fill in the blank multiple times, thereby obtaining multiple fill-in-the-blank answers for one fill-in-the-blank question. In the example shown in Figure 5, three fill-in-the-blank answers are recorded for one fill-in-the-blank question.

[0064] In addition, a fill-in-the-blank question may have multiple comparison targets (fill-in-the-blank locations). When there are multiple comparison targets, the serial numbers may be incremented and recorded in the fill-in-the-blank history information 33, for example, $1=..., $2=..., $3=...

[0065] In the example shown below, for simplicity, a fill-in-the-blank question with one comparison target for the AI ​​answer is shown.

[0066] The semantic analysis unit 27 extracts triplet pairs, hypernyms, and descriptive languages ​​using the AI ​​answers read from the dialogue history information 32 as input, and outputs the hypernyms and languages ​​(descriptive languages) for each triplet. The semantic analysis unit 27 may particularly output the hypernyms and languages ​​of the object words. The object words included in the AI ​​answer (first sentence) are an example of first words.

[0067] The semantic analysis unit 27 may extract the hypernym and language of the object, for example, by using the first semantic analysis method described below.

[0068] [First Semantic Analysis Method] FIG. 6 is a diagram for explaining the first semantic analysis method by the semantic analysis unit 27 of the hallucination detection device 1 according to the first embodiment.

[0069] The first semantic analysis method includes the following steps 1 to 3.

[0070] Process 1: The semantic analysis unit 27 extracts triplet (subject, predicate, object) from the sentence of the AI ​​answer using, for example, a known knowledge graph extractor.

[0071] 6, the AI ​​answer is "XX Company's headquarters is Tokyo." The semantic analysis unit 27 uses a knowledge graph extractor to extract "XX Company's headquarters" as the subject, "desu" as the predicate, and "Tokyo" as the object from this AI answer "XX Company's headquarters is Tokyo."

[0072] Process 2: The semantic analysis unit 27 associates the object with its hypernym, for example, using a known thesaurus (for example, WordNet, Japanese Thesaurus).

[0073] In the example shown in FIG. 6, the semantic analysis unit 27 determines the superordinate word "prefecture" from the object "Tokyo" and associates this "prefecture" with "Tokyo."

[0074] Process 3: The semantic analysis unit 27 determines the description language of the text of the AI ​​response, for example, using a known regular expression technique.

[0075] For example, if the sentence of the AI ​​response matches the regular expression [a-n]|[a-vu] for "hiragana or katakana", the semantic analysis unit 27 may determine that the written language of the sentence of the AI ​​response is Japanese.

[0076] The semantic analysis unit 27 obtains an extraction result using the first semantic analysis method described above. In the example shown in Fig. 6, the extraction result includes the AI ​​response "The head office of XX Company is Tokyo" as [Sentence 1], as well as "Subject = Head Office of XX Company," "Predicate = "desu," and "Object = Tokyo." Furthermore, the extraction result shown in Fig. 6 also includes "Hyperword of Object = Prefecture" and "Language of Object = Japanese."

[0077] Furthermore, the semantic analysis unit 27 may use the second semantic analysis method described below instead of the first semantic analysis method described above to output the hypernym of the object and the language.

[0078] [Second Semantic Analysis Method] FIG. 7 is a diagram for explaining a second semantic analysis method by the semantic analysis unit 27 of the hallucination detection device 1 according to the first embodiment.

[0079] In the second semantic analysis method, the semantic analysis unit 27 uses an LLM to simultaneously extract the hypernyms and language of the object from the sentence of the AI ​​answer. The LLM used by the semantic analysis unit 27 may be different from or the same as the LLM used by the interactive generation AI that output the AI ​​answer. The semantic analysis unit 27 generates a prompt (extraction prompt) to extract "triplets (subject, predicate, object)" and "hypernyms and languages ​​of each object" from the sentence of the AI ​​answer (see symbol P1 in FIG. 7).

[0080] In the extraction prompt shown in Figure 7, a delimiter (===) is placed below the prompt sentence, "Extract the subject, predicate, and object from the following sentence, as well as the hypernym and language for each object." Below this delimiter, the AI ​​response "XX Company's headquarters is in Tokyo" is placed as [Sentence 1]. Note that the format of the extraction prompt is not limited to the example shown in Figure 7 and can be modified as appropriate.

[0081] The semantic analysis unit 27 inputs the created extraction prompt into the LLM and outputs the extraction result. The semantic analysis unit 27 obtains the extraction result using the second semantic analysis method described above. In the example shown in FIG. 7 , similar to the example shown in FIG. 6 , the extraction result includes the AI ​​response "The headquarters of XX Company is Tokyo" as [Sentence 1], as well as "Subject = Head Office of XX Company," "Predicate = "desu," and "Object = Tokyo." Furthermore, the extraction result shown in FIG. 7 also includes "Hyperword of Object = Prefecture" and "Language of Object = Japanese."

[0082] The hypernym and the description language are examples of attributes of the first word included in the AI ​​answer (first sentence).

[0083] The semantic analysis unit 27 stores the extraction results in a predetermined storage area or the like of the storage device 13 .

[0084] The fill-in-the-blank question generation unit 24 generates fill-in-the-blank questions. The fill-in-the-blank questions are used as prompts to be input into the LLM. The fill-in-the-blank questions include a portion of the sentence in the AI ​​answer that corresponds to the object as a "fill-in-the-blank portion." The fill-in-the-blank portion is an example of an input portion generated according to an attribute.

[0085] In the present hallucination detection device 1, the fill-in-the-blank answer unit 25, which will be described later, inputs fill-in-the-blank questions into the LLM, and causes the subject to answer by entering an object corresponding to the fill-in-the-blank portion.

[0086] FIG. 8 is a diagram for explaining the processing of the fill-in-the-blank question generation unit 24 of the hallucination detection device 1 according to the first embodiment.

[0087] The fill-in-the-blank question generation unit 24 generates a variable name corresponding to each object for one or more objects in the AI ​​answer extracted by the semantic analysis unit 27. For example, the fill-in-the-blank question generation unit 24 may obtain the hypernym and language of the object from the extraction result generated by the semantic analysis unit 27, and use these hypernyms and languages ​​of the object to generate a variable name in the format of, for example, "$hypernym_language_serial number".

[0088] The serial number is used to identify the object, for example, when the same object appears in multiple positions, it indicates that the object in each position is the same.

[0089] 8, the object is Tokyo, the hypernym is prefecture, and the language is Japanese. The fill-in-the-blank question generation unit 24 generates a variable name "$prefecture_Japanese_1" for the object = Tokyo.

[0090] The fill-in-the-blank question includes an instruction to the LLM. In the example shown in Figure 8, the fill-in-the-blank question includes the instruction "Answer the next fill-in-the-blank question" (see symbol P1 in Figure 8).

[0091] Furthermore, the fill-in-the-blank question generation unit 24 generates a fill-in-the-blank sentence by replacing the object of the sentence in the AI ​​answer with a variable name. The fill-in-the-blank question generation unit 24 inserts the generated fill-in-the-blank sentence into the fill-in-the-blank question.

[0092] In the example shown in Figure 8, the fill-in-the-blank sentence is "The head office of XX Company is $Prefecture_Japanese_1" (see symbol P2 in Figure 8), and "$Prefecture_Japanese_1" contained in this fill-in-the-blank sentence is the variable name (see symbol P3 in Figure 8).

[0093] Furthermore, the fill-in-the-blank question generation unit 24 generates a hint to be shown in the fill-in-the-blank question together with the instruction and the fill-in-the-blank sentence. The hint may be a sentence showing how to read the variable name. The fill-in-the-blank question generation unit 24 inserts the created hint into the fill-in-the-blank question at a position following the instruction "Answer the next fill-in-the-blank question."

[0094] In the example shown in FIG. 8, the hint is "However, in the fill-in-the-blanks, please write the hypernym and language you expect in the answer in the format of '$hypernym_language_serial number'" (see symbol P4 in FIG. 8).

[0095] The hint is an example of an instruction that prompts the user to input an answer that satisfies the attribute in a fill-in-the-blank section (input section).

[0096] When a fill-in-the-blank question is input, the LLM refers to this hint and understands that the object corresponding to the fill-in-the-blank in the fill-in-the-blank sentence is a hyponym of "prefecture" and that the answer should be in Japanese.

[0097] The fill-in-the-blank question generator 24 dynamically adds hints to the fill-in-the-blank questions. The hints may be called dynamic hints.

[0098] In addition, the fill-in-the-blank question generation unit 24 may place a delimiter between the above-mentioned instruction statement and hint and the fill-in-the-blank sentence in a fill-in-the-blank question. In the fill-in-the-blank question illustrated in Fig. 8, the delimiter "====" is placed below the hint (see symbol P5 in Fig. 8), and the fill-in-the-blank sentence is placed below this delimiter.

[0099] The delimiter is not limited to "====" and can be changed as appropriate. The delimiter may be configured to have the number of characters equivalent to one token. A simple line break may also be used as the delimiter.

[0100] The fill-in-the-blank question generating unit 24 causes the fill-in-the-blank storage processing unit 23 to store the created fill-in-the-blank questions in the fill-in-the-blank history information 33 .

[0101] The fill-in-the-blank storage processor 23 registers the fill-in-the-blank question and the comparison object in the fill-in-the-blank history information 33 in accordance with instructions from the fill-in-the-blank question generator 24. The fill-in-the-blank storage processor 23 records the fill-in-the-blank answer and the comparison object (the object to be compared) in the format of "$serial number=blanked object" in the comparison object column of the fill-in-the-blank history information 33 (see FIG. 5).

[0102] FIG. 9 is a diagram showing the process of registering fill-in-the-blank questions generated by the fill-in-the-blank question generating unit 24 in the fill-in-the-blank history information 33 in the hallucination detection device 1 according to the first embodiment.

[0103] 9 shows an example in which fill-in-the-blank questions generated by the fill-in-the-blank question generation unit 24 are stored in a fill-in-the-blank question sequence in association with the AI ​​answer in the fill-in-the-blank history information 33. In addition, the fill-in-the-blank history information 33 stores a comparison object ($serial number = blanked object) in association with this fill-in-the-blank question.

[0104] The fill-in-the-blank answer unit 25 requests the LLM to fill in the blanks multiple times using the fill-in-the-blank questions generated by the fill-in-the-blank question generation unit 24 as prompts, and collects multiple answers. The fill-in-the-blank answer unit 25 also causes the fill-in-the-blank storage processing unit 23 to register the multiple answers obtained from the LLM in the fill-in-the-blank history information 33. That is, the fill-in-the-blank answer unit 25 registers the multiple answers obtained from the LLM in the fill-in-the-blank history information 33 via the fill-in-the-blank storage processing unit 23.

[0105] FIG. 10 is a diagram for explaining the processing of the fill-in-the-blank answer unit 25 of the hallucination detection device 1 according to the first embodiment.

[0106] The fill-in-the-blank answer unit 25 reads a fill-in-the-blank question from the fill-in-the-blank history information 33 and inputs the read fill-in-the-blank question into the LLM multiple times. In the example shown in Fig. 10, the fill-in-the-blank answer unit 25 inputs one fill-in-the-blank question read from the fill-in-the-blank history information 33 into the LLM three times.

[0107] Then, the fill-in-the-blank answer unit 25 registers the multiple answers to the fill-in-the-blank questions obtained from the LLM in the fill-in-the-blank history information 33 via the fill-in-the-blank storage processing unit 23. The answers obtained from the LLM to the fill-in-the-blank questions may be called fill-in-the-blank answers.

[0108] The LLM refers to the hints contained in the fill-in-the-blank question generated by the fill-in-the-blank question generation unit 24 and outputs a fill-in-the-blank answer in Japanese that is a hyponym of "prefecture" for the blank portion in the fill-in-the-blank sentence.

[0109] In the example shown in Figure 10, three fill-in-the-blank answers, "$1=Tokyo", "$1=Kanagawa", and "$1=Tokyo", are obtained from the LLM, and the fill-in-the-blank answer unit 25 causes the fill-in-the-blank storage processing unit 23 to register these three fill-in-the-blank answers in the fill-in-the-blank history information 33.

[0110] The hallucination presence / absence determination unit 26 determines the presence or absence of hallucinations based on a plurality of fill-in-the-blank answers (a plurality of answers) to the fill-in-the-blank questions from the LLM.

[0111] FIG. 11 is a diagram for explaining the processing of the hallucination presence / absence determination unit 26 of the hallucination detection device 1 according to the first embodiment.

[0112] The hallucination presence / absence determination unit 26 acquires a plurality of fill-in-the-blank answers from the fill-in-the-blank history information 33. The plurality of fill-in-the-blank answers acquired from the fill-in-the-blank history information 33 are an example of a plurality of second words corresponding to a first word.

[0113] The hallucination presence / absence determination unit 26 calculates the distance from the comparison target (object) for each fill-in-the-blank answer. For example, if the comparison target and the fill-in-the-blank answer are a perfect match, the distance may be set to 0. For example, if the comparison target is "$1=Tokyo" and the fill-in-the-blank answer is "$1=Tokyo", the distance is set to 0.

[0114] Furthermore, if the comparison target is included in the fill-in-the-blank answer, the distance may be set to 0. For example, if the comparison target is "$1 = Tokyo" and the fill-in-the-blank answer is "$1 = Tokyo," the distance is set to 0.

[0115] Furthermore, if the fill-in-the-blank answer is included in the comparison target, the distance may be set to 0.5. For example, if the comparison target is "$1 = central Tokyo" and the fill-in-the-blank answer is "$1 = Tokyo," the distance is set to 0.5.

[0116] In other cases, the distance can be calculated as 1. Also, the distance between the fill-in-the-blank answer and the comparison object (object) can be simply called the distance of the fill-in-the-blank answer.

[0117] The hallucination presence / absence determination unit 26 may determine the presence or absence of a hallucination based on the distances between a plurality of fill-in-the-blank answers. For example, if the average value of the distances between a plurality of fill-in-the-blank answers is equal to or greater than a threshold, the hallucination presence / absence determination unit 26 may determine that the comparison target is a hallucination.

[0118] For example, as shown in Figure 11, if the comparison target is "$1 = Tokyo" and the three fill-in-the-blank answers obtained are "$1 = Tokyo", "$1 = Kanagawa", and "$1 = Tokyo", the average value of the distances between the multiple fill-in-the-blank answers can be calculated using the following formula.

[0119] Average distance = (0 + 1 + 0) / 3 ≈ 0.33 Here, when the threshold is 0.5, the average value of the distances of the multiple fill-in-the-blank answers, 0.33, is less than the threshold of 0.5, so the hallucination presence / absence determination unit 26 determines that it is not a hallucination. The hallucination presence / absence determination unit 26 may output a hallucination presence / absence explanation indicating that it is not a hallucination as the determination result (see symbol P1 in FIG. 11 ).

[0120] In another example, if the comparison target is "$1 = Shizuoka" and the fill-in-the-blank answers are "$1 = Tokyo", "$1 = Kanagawa", "$1 = Tokyo", the average distance of the multiple answers can be calculated using the following formula:

[0121] Average distance = (1 + 1 + 1) / 3 ≈ 1.00 Here, when the threshold is 0.5, the average value of the distances of the multiple fill-in-the-blank answers, 1.00, is greater than or equal to the threshold of 0.5, so the hallucination presence / absence determination unit 26 determines that it is a hallucination. The hallucination presence / absence determination unit 26 may output a hallucination presence / absence explanation indicating that it is a hallucination as the determination result.

[0122] The hallucination determination unit 26 determines whether the AI ​​answer (first sentence) is a hallucination by comparing multiple fill-in-the-blank answers (second words) corresponding to the object (first word) contained in the multiple fill-in-the-blank answers with the comparison object (first word).

[0123] In addition, the hallucination presence / absence determination unit 26 may tally up the frequencies (occurrence frequencies) of multiple fill-in-the-blank answers and output this as an explanatory statement along with the determination result of whether or not a hallucination occurred. In the example indicated by symbol P1 in Fig. 11, the number of times "$1 = Tokyo" was given as the fill-in-the-blank answer (Tokyo = 2 times) and the number of times "$1 = Kanagawa" was given as the fill-in-the-blank answer (Kanagawa = 1 time) are shown.

[0124] The frequency (occurrence frequency) of multiple fill-in-the-blank answers is an example of a count result of multiple answers to a fill-in-the-blank question (second sentence) output by the LLM (predetermined model).

[0125] Furthermore, in this hallucination detection device 1, a state in which the presence or absence of a hallucination is correctly detected may be referred to as an OK phenomenon, and a state in which the presence or absence of a hallucination is not correctly detected may be referred to as an NG phenomenon.

[0126] (C) Operation The processing of the hallucination detection device 1 according to the first embodiment configured as described above will be described in accordance with the flowchart (steps A1 to A6) shown in Fig. 12, with reference to Fig. 13. Fig. 13 is a diagram for explaining the processing of the hallucination detection device 1 according to the first embodiment.

[0127] In step A1, the dialogue storage processor 22 initializes (clears) the records (rows) of the dialogue history information 32, and the blank filling processor 23 initializes (clears) the records (rows) of the blank filling history information 33. In addition, the LLM storage processor 21 registers the usage information of the LLM to be used in the LLM information 31.

[0128] These processes initialize each piece of information (LLM information 31, dialogue history information 32, and blank-filling history information 33). The dialogue-based generation AI outputs an AI answer to the user's question (see symbol B1 in FIG. 13).

[0129] In step A2, the dialogue storage processing unit 22 records (adds) the user question and the AI ​​response input via the input / output device 15 to the dialogue history information 32.

[0130] In step A3, the semantic analysis unit 27 uses the AI ​​response read from the dialogue history information 32 as input and performs triplet extraction, hypernym extraction, and descriptive language extraction to extract the hypernym and language of the object word (see symbol B2 in Figure 13).

[0131] In step A4, the fill-in-the-blank question generation unit 24 generates a fill-in-the-blank question (see reference symbol B4 in FIG. 13). The fill-in-the-blank question generation unit 24 includes a directive, a fill-in-the-blank sentence, and a hint in the fill-in-the-blank question.

[0132] In this case, the fill-in-the-blank question generation unit 24 acquires the hypernym of the object and the language from the extraction result generated by the semantic analysis unit 27, and generates a variable name using these hypernym of the object and the language. Then, the fill-in-the-blank question generation unit 24 creates a fill-in-the-blank sentence in which a fill-in portion (blank) is formed by replacing the object with the variable name in the sentence of the AI ​​answer (see symbol B3 in FIG. 13). The hint may be a sentence showing how to read the variable name.

[0133] In step A5, the fill-in-the-blank answer unit 25 requests the LLM to fill in the blanks multiple times using the fill-in-the-blank questions generated by the fill-in-the-blank question generation unit 24 as prompts (see symbol B5 in Figure 13), and collects multiple answers (see symbol B6 in Figure 13).

[0134] In step A6, the hallucination presence / absence determination unit 26 determines whether or not a hallucination is present based on the object (comparison object) and a plurality of fill-in-the-blank answers to the fill-in-the-blank questions from the LLM.

[0135] For example, the hallucination presence / absence determination unit 26 compares the object (comparison target) with each fill-in-the-blank answer and calculates the distance between each fill-in-the-blank answer and the comparison target (object).The hallucination presence / absence determination unit 26 then compares the average value of the distances between the comparison target (object) and the multiple fill-in-the-blank answers with a threshold to determine whether or not a hallucination is occurring.The hallucination presence / absence determination unit 26 outputs the determination result and presents it to the user (see symbol B7 in FIG. 13).Then, the processing ends.

[0136] (D) Effects As described above, according to the hallucination detection device 1 according to the first embodiment, the fill-in-the-blank question generation unit 24 generates a fill-in-the-blank question that includes a directive, a fill-in-the-blank sentence, and a hint.

[0137] Specifically, the semantic analysis unit 27 extracts triplet pairs, hypernyms, and descriptive languages ​​using the AI ​​answer as input, and outputs the hypernym and language of the object. The fill-in-the-blank question generation unit 24 generates a variable name in the format "$hypernym_language_serial number" using the hypernym and language of the object, and creates a fill-in-the-blank sentence by replacing the object in the sentence of the AI ​​answer with this variable name.

[0138] Furthermore, the fill-in-the-blank question generation unit 24 generates hints indicating how to read the variable names, for example, and includes them in the fill-in-the-blank questions together with the directives and fill-in-the-blank statements.

[0139] Then, the fill-in-the-blank answer unit 25 requests the LLM to fill in the blanks multiple times using the fill-in-the-blank questions generated by the fill-in-the-blank question generation unit 24 as prompts, and collects multiple answers.

[0140] In fill-in-the-blank questions (prompts), by dynamically including hints in the fill-in-the-blank portions of the fill-in-the-blank sentence, the directionality of multiple fill-in-the-blank answers by the LLM can be aligned according to the attributes of the AI ​​answer. This makes it more appropriate to determine whether or not a hallucination exists based on the character string distance between the object (comparison object) and the fill-in-the-blank answer. As a result, it is possible to reduce false positive cases in which a non-hallucinatory object is mistaken for a hallucination, thereby improving the accuracy of hallucination detection. Furthermore, in this case, hallucination detection can be achieved with high accuracy without narrowing the domain of the AI ​​answer.

[0141] FIG. 14 is a diagram for explaining a false positive case in which a non-hallucinatory object is mistaken for a hallucination.

[0142] In the fill-in-the-blank question shown in FIG. 14, the fill-in-the-blank sentence (fill-in-the-blank question) is created by simply using the object (Tokyo) included in the sentence of the AI ​​response as the fill-in-the-blank, and no substitution using a variable name in the fill-in-the-blank sentence or addition of a hint is performed. Therefore, the LLM cannot know that the fill-in-the-blank is a hyponym of "prefecture" and that an answer in Japanese is required. Therefore, when the fill-in-the-blank question shown in FIG. 14 is input, the LLM outputs three answers as fill-in-the-blank answers, for example, "$1=building," "$1=Minato Ward," and "$1=Tokyo." All of these fill-in-the-blank answers are considered to be correct as objects representing the headquarters of XX company.

[0143] Here, the distance between each fill-in-the-blank answer and the comparison target (object) is calculated, as in the example shown in Figure 11. If the comparison target and the answer are a perfect match, the distance is set to 0; if the answer is included in the comparison target, the distance is set to 0.5; and in all other cases, the distance is set to 1.

[0144] As shown in Figure 13, if the comparison target is "$1 = Tokyo" and the three fill-in-the-blank answers obtained are "$1 = building," "$1 = Minato Ward," and "$1 = Tokyo," the average value of the distances between the multiple answers can be calculated using the following formula.

[0145] Average distance = (1 + 1 + 10) / 3 ≒ 1.00 Here, if the threshold is 0.5, the average distance of multiple answers, 1.00, is above the threshold of 0.5 and is therefore judged to be a hallucination. The three answers (fill-in-the-blank answers) "$1 = building," "$1 = Minato Ward," and "$1 = Tokyo" are all correct as objects, but are nonetheless judged to be hallucinations.

[0146] For the comparison object "$1 = Tokyo," the fill-in-the-blank answers "$1 = building," "$1 = Minato Ward," and "$1 = Tokyo" are correct, but they can be said to be answers that deviate from the direction of the fill-in-the-blank answer. It is expected that there will be multiple directions for fill-in-the-blank answers, and it is difficult to compare answers that differ in such direction.

[0147] Generally, if the direction of the answer is not specified, multiple fill-in-the-blank answers tend to have different expressions in terms of "type / unit," "specificity," and "language." For the fill-in-the-blank "$1 = Tokyo," "$1 = building" is a different type, "$1 = Minato Ward" is a different specificity, and "$1 = Tokyo" is a different language. Also, for example, for the fill-in-the-blank "$1 = 365," "$1 = 1 year" is a different unit, "$1 = hundreds" is a different specificity, and "$1 = Three hundreds" is a different language.

[0148] In this hallucination detection device 1, even when there are multiple directions (types, languages, etc.) of fill-in-the-blank answers, the dynamically added hints can align (limit) the directions of the fill-in-the-blank answers, with only a slight increase in the load of the semantic analysis unit 27 extracting superordinate words and languages ​​from the object, thereby improving the accuracy of hallucination detection.

[0149] (II) Description of the Second Embodiment In the first embodiment described above, a case where the AI ​​response is an affirmative sentence has been described. In the second embodiment, a case where the AI ​​response is a negative sentence will be described.

[0150] Hallucination detection in negative sentences cannot be achieved by comparing the object's re-asking and the AI's answer. This is because almost all answers will be correct, so if the multiple answers obtained by re-asking are inconsistent, it is invalid to judge that it is a hallucination. Also, if the multiple answers obtained by re-asking are consistent, it is invalid to judge that it is not a hallucination.

[0151] For example, consider a case where the AI ​​responds to a user question, "Where is the headquarters of XX Company?" with a negative sentence, "XX Company's headquarters is not Shizuoka." In such a case, the fill-in-the-blank question, which uses the object "Shizuoka" in the AI ​​response as the fill-in-the-blank part, would be "XX Company's headquarters is not $1."

[0152] Here, if the headquarters of X company is "Tokyo," all fill-in-the-blank answers except "Tokyo" will be correct, and even if the multiple answers obtained by asking again are different, it cannot be said to be a hallucination. Also, if the multiple answers obtained by asking again are consistent with "Tokyo," it must be said to be a hallucination. In this way, if the AI ​​answer is a negative sentence, the judgment of whether or not there is a hallucination can be reversed (although it does not necessarily reverse). Also, the basis for the judgment is invalid.

[0153] Therefore, if the AI ​​response is a negative sentence, it is difficult for the hallucination detection device 1 according to the first embodiment described above to determine whether or not a hallucination is occurring.

[0154] In the second embodiment, when the semantic analysis unit 27 extracts triplet pairs, it also extracts the polarity of the sentence (positive or negative), and if the sentence is negative, the correctness of the sentence and the reason for the correctness judgment (reason for the correctness judgment) are used as fill-in-the-blanks and are automatically added to the prompt along with a fill-in-the-blank hint.

[0155] (A) Configuration Example The hallucination detection device 1 according to the second embodiment has the same functional and hardware configuration as the first embodiment, as shown in Figures 1 and 2. In the following figures, the same reference numerals as those already described indicate the same parts, and therefore their explanation will be omitted.

[0156] FIG. 15 is a diagram illustrating an example of the data structure of the dialogue history information 32 in the hallucination detection device 1 according to the second embodiment.

[0157] In the hallucination detection device 1 according to the second embodiment, as shown in FIG. 15, not only positive sentences (see serial number: 1) but also negative sentences (see serial number: 2) are registered in the AI ​​responses of the dialogue history information 32.

[0158] In the example shown in Figure 15, in serial number 2 of the dialogue history information 32, for example, a user question "The headquarters of XX Company is in Shizuoka, right?" is recorded in association with the AI ​​answer "The headquarters of XX Company is not in Shizuoka."

[0159] The dialogue storage processing unit 22 acquires, for example, a user question and an AI response to the user question input via the input / output device 15 , and records them in the dialogue history information 32 .

[0160] FIG. 16 is a diagram illustrating an example of the data structure of the filling history information 33 in the hallucination detection device 1 according to the second embodiment.

[0161] In the hallucination detection device 1 according to the second embodiment, not only positive sentences but also negative sentences are registered in the AI ​​responses of the fill-in history information 33, as shown in FIG.

[0162] In the hallucination detection device 1 according to the second embodiment, the semantic analysis unit 27 receives the AI ​​response read from the dialogue history information 32 as input, performs triplet extraction and polarity extraction, and outputs the triplet and the polarity of the sentence.

[0163] The semantic analysis unit 27 may extract the triplet and the polarity of the sentence using, for example, the first semantic analysis method described below.

[0164] [First Semantic Analysis Method] FIG. 17 is a diagram for explaining the first semantic analysis method by the semantic analysis unit 27 of the hallucination detection device 1 according to the second embodiment.

[0165] The first semantic analysis method includes the following processes 1 and 2.

[0166] Process 1: The semantic analysis unit 27 extracts triplet (subject, predicate, object) from the sentence of the AI ​​answer using, for example, a known knowledge graph extractor.

[0167] 17, the AI ​​answer is "XX Company's headquarters is not Shizuoka." Using a knowledge graph extractor, the semantic analysis unit 27 extracts "XX Company's headquarters" as the subject, "is not" as the predicate, and "Shizuoka" as the object from this AI answer "XX Company's headquarters is not Shizuoka."

[0168] Process 2: The semantic analysis unit 27 uses, for example, a known negative sentence recognizer to determine whether the sentence is a negative sentence or an affirmative sentence.

[0169] The semantic analysis unit 27 may determine whether a sentence is a negative sentence or an affirmative sentence based on, for example, the presence or absence of negative words, negative expressions, indirect negative expressions, etc. in the AI ​​response. Negative words, negative words, negative expressions, indirect negative expressions, etc. contained in the sentence of the AI ​​response are examples of information indicating whether or not the sentence is a negative sentence.

[0170] In the example shown in FIG. 17, the semantic analysis unit 27 determines that "Fujitsu's head office is not in Shizuoka" is a negative sentence, and outputs "sentence polarity = negative."

[0171] The semantic analysis unit 27 obtains an extraction result by the first semantic analysis method described above. In the example shown in Fig. 17, the extraction result includes the AI ​​response "The headquarters of XX Company is not Shizuoka" as [Sentence 1], as well as "Subject = XX Company Headquarters," "Predicate = is not," and "Object = Shizuoka." Furthermore, the extraction result shown in Fig. 17 also includes "Sentence Polarity = Negative."

[0172] Furthermore, the semantic analysis unit 27 may use the second semantic analysis method described below instead of the first semantic analysis method described above to output the hypernym of the object and the language.

[0173] [Second Semantic Analysis Method] FIG. 18 is a diagram for explaining a second semantic analysis method by the semantic analysis unit 27 of the hallucination detection device 1 according to the second embodiment.

[0174] In the second semantic analysis method, the semantic analysis unit 27 uses an LLM to simultaneously extract triplets and sentence polarities from the sentence of the AI ​​answer. The LLM used by the semantic analysis unit 27 may be different from or the same as the LLM used by the interactive generation AI that output the AI ​​answer. The semantic analysis unit 27 generates a prompt (extraction prompt) that extracts "triplets (subject, predicate, object)" and "polarity (negative or affirmative)" from the sentence of the AI ​​answer (see symbol P1 in FIG. 18).

[0175] In the extraction prompt shown in Figure 18, a delimiter (===) is placed below the prompt statement "Extract the subject, predicate, object, and polarity (negative or affirmative) of each sentence from the following sentence," and below this delimiter is placed the AI ​​response "The headquarters of XX Company is not in Shizuoka" as [Sentence 1]. Note that the format of the extraction prompt is not limited to the example shown in Figure 18 and can be modified as appropriate.

[0176] The semantic analysis unit 27 inputs the created extraction prompt into the LLM and outputs the extraction result. The semantic analysis unit 27 obtains the extraction result using the second semantic analysis method described above. In the example shown in FIG. 18 , similar to the example shown in FIG. 17 , the extraction result includes the AI ​​response “The headquarters of XX Company is not Shizuoka” as [Sentence 1], as well as “Subject = XX Company Headquarters,” “Predicate = is not,” and “Object = Shizuoka.” Furthermore, the extraction result shown in FIG. 18 also includes “Sentence Polarity = Negative.” Sentence polarity is an example of an attribute of the AI ​​response (first sentence).

[0177] The semantic analysis unit 27 stores the extraction results in a predetermined storage area or the like of the storage device 13 .

[0178] The fill-in-the-blank question generation unit 24 generates fill-in-the-blank questions. The fill-in-the-blank questions are used as prompts to be input into the LLM. In the present hallucination detection device 1, the fill-in-the-blank answer unit 25, which will be described later, inputs the fill-in-the-blank questions into the LLM and causes the LLM to answer with an object corresponding to the fill-in part. The fill-in-the-blank question is an example of a query to be input into the LLM (predetermined model) corresponding to the AI ​​answer (first sentence), and is an example of the second sentence.

[0179] When the AI ​​answer (first sentence) is a negative sentence, the fill-in-the-blank question generation unit 24 generates a fill-in-the-blank question (second sentence) in which the result of the correctness judgment of the AI ​​answer is used as the fill-in-the-blank section (input section).

[0180] FIG. 19 is a diagram for explaining the processing of the fill-in-the-blank question generation unit 24 of the hallucination detection device 1 according to the second embodiment.

[0181] The fill-in-the-blank question generation unit 24 generates two variables corresponding to the negative sentences in the AI ​​answer, for example, in the format of "$reason_serial number" and "$correct / incorrect judgment_serial number".

[0182] The fill-in-the-blank questions include instructions to the LLM. In the example shown in Figure 19, the fill-in-the-blank questions include the instruction "Answer the next fill-in-the-blank question" (see symbol P1 in Figure 19).

[0183] Furthermore, the fill-in-the-blank question generation unit 24 adds to the fill-in-the-blank question a fill-in-the-blank sentence that asks whether the negative sentence in the AI ​​answer is true or false and the reason for that true or false judgment. In this fill-in-the-blank sentence, the variable "$true or false judgment_serial number" created as described above is used as the fill-in-the-blank portion that asks about the true or false judgment, and the similarly created variable "$reason_serial number" is used as the fill-in-the-blank portion that asks about the reason for the true or false judgment.

[0184] That is, the fill-in-the-blank question generating unit 24 realizes the fill-in-the-blank portion by expressing the reason and the correctness judgment in this fill-in-the-blank sentence with the variables "$reason_serial number" and "$correctness judgment_serial number."

[0185] In the example shown in Fig. 19, the negative sentence of the AI ​​answer is "The headquarters of XX Company is not in Shizuoka." In this case, the fill-in-the-blank question generation unit 24 adds a fill-in-the-blank sentence "[Sentence 1] Because $Reason_1, therefore, 'The headquarters of XX Company is not in Shizuoka' is $True / False Judgment_2" to the fill-in-the-blank question (see symbol P2 in Fig. 19). That is, the fill-in-the-blank sentence includes "$Reason_Serial Number" and "$True / False Judgment_Serial Number" (see symbols P3 and P4 in Fig. 19).

[0186] In addition, the fill-in-the-blank question generation unit 24 adds a hint to the fill-in-the-blank question about how to solve the variable "$true / false judgment_serial number." In the example shown in Fig. 19, the hint "However, for $true / false judgment_2, select either "true" or "false"" is shown below the instruction (see symbol P5 in Fig. 19).

[0187] The fill-in-the-blank question generation unit 24 may place a delimiter between the above-mentioned instruction sentence and hint and the fill-in-the-blank sentence in a fill-in-the-blank question. In the fill-in-the-blank question illustrated in Fig. 19, the delimiter "====" is placed below the hint (see symbol P6 in Fig. 19), and the fill-in-the-blank sentence is placed below this delimiter.

[0188] The delimiter is not limited to "====" and can be changed as appropriate. The delimiter may be configured to have the number of characters equivalent to one token. A simple line break may also be used as the delimiter.

[0189] The fill-in-the-blank question generating unit 24 causes the fill-in-the-blank storage processing unit 23 to store the created fill-in-the-blank questions in the fill-in-the-blank history information 33 .

[0190] The fill-in-the-blank storage processor 23 registers the fill-in-the-blank questions and the comparison targets in the fill-in-the-blank history information 33 in accordance with instructions from the fill-in-the-blank question generator 24. The fill-in-the-blank storage processor 23 records the fill-in-the-blank answer and the comparison target (the correctness judgment result to be compared) in the format of "$serial number=correctness judgment result" in the comparison target column of the fill-in-the-blank history information 33 (see FIG. 16). The comparison target is an example of correctness judgment reference information.

[0191] FIG. 20 is a diagram showing the process of registering fill-in-the-blank questions generated by the fill-in-the-blank question generating unit 24 in the fill-in-the-blank history information 33 in the hallucination detection device 1 according to the second embodiment.

[0192] 20 shows an example in which fill-in-the-blank questions generated by the fill-in-the-blank question generation unit 24 are stored in a fill-in-the-blank question sequence in association with the AI ​​answer in the fill-in-the-blank history information 33. In addition, in the fill-in-the-blank history information 33, a comparison target ($true / false judgment_2) is stored in association with this fill-in-the-blank question.

[0193] In the example shown in FIG. 20, the fill-in-the-blank storage processing unit 23 wants to determine the accuracy of the assertion that "the AI ​​answer is correct," so it records "$serial number = correct" as the comparison target. If the majority of the fill-in-the-blank answers are "correct," it can be determined that "it is not a hallucination," and if not, it can be determined that "it is a hallucination." Therefore, if consistency is required in the reasons for determining accuracy, the fill-in-the-blank storage processing unit 23 may include "$serial number_default reason" as the comparison target.

[0194] The fill-in-the-blank answer unit 25 requests the LLM to fill in the blanks multiple times using the fill-in-the-blank questions generated by the fill-in-the-blank question generation unit 24 as prompts, and collects multiple answers. The fill-in-the-blank answer unit 25 also causes the fill-in-the-blank storage processing unit 23 to register the multiple answers obtained from the LLM in the fill-in-the-blank history information 33. That is, the fill-in-the-blank answer unit 25 registers the multiple answers obtained from the LLM in the fill-in-the-blank history information 33 via the fill-in-the-blank storage processing unit 23.

[0195] FIG. 21 is a diagram for explaining the processing of the fill-in-the-blank answer unit 25 of the hallucination detection device 1 according to the second embodiment.

[0196] The fill-in-the-blank answer unit 25 reads a fill-in-the-blank question from the fill-in-the-blank history information 33 and inputs the read fill-in-the-blank question into the LLM multiple times. In the example shown in Fig. 21, the fill-in-the-blank answer unit 25 inputs one fill-in-the-blank question read from the fill-in-the-blank history information 33 into the LLM three times.

[0197] Then, the fill-in-the-blank answer unit 25 registers the multiple answers to the fill-in-the-blank questions obtained from the LLM in the fill-in-the-blank history information 33 via the fill-in-the-blank storage processing unit 23.

[0198] The LLM refers to the hints contained in the fill-in-the-blank questions generated by the fill-in-the-blank question generation unit 24, and outputs the reason for the correctness judgment and the correctness judgment result ("correct" or "incorrect") as the fill-in-the-blank answer for the fill-in part in the fill-in-the-blank sentence.

[0199] In the example shown in FIG. 21 , three fill-in-the-blank answers are obtained from the LLM: “$1 = The head office of XX Company is in Tokyo, so $2 = correct”, “$1 = The head office of XX Company is not in Shizuoka, so $2 = correct”, and “$1 = XX Company is a global company, so $2 = incorrect”, and the fill-in-the-blank answer unit 25 causes the fill-in-the-blank storage processing unit 23 to register these three fill-in-the-blank answers in the fill-in-the-blank history information 33.

[0200] In the LLM, even if hints are given, there is a probability that a wrong answer will be given. In the example shown in Figure 21, the fill-in-the-blank answer "$1 = XX company is a global company, $2 = wrong" is an incorrect answer.

[0201] The hallucination presence / absence determination unit 26 determines whether or not a hallucination occurs based on a plurality of fill-in-the-blank answers to fill-in-the-blank questions from the LLM.

[0202] FIG. 22 is a diagram for explaining the processing of the hallucination presence / absence determination unit 26 of the hallucination detection device 1 according to the second embodiment.

[0203] The hallucination presence / absence determination unit 26 acquires the fill-in answer from the fill-in history information 33. The hallucination presence / absence determination unit 26 calculates the distance from the comparison target (object word) for each fill-in answer according to the hypernym.

[0204] For example, if the comparison target and the fill-in-the-blank answer are a perfect match, the distance may be set to 0. For example, if the comparison target is "$2=correct" and the fill-in-the-blank answer is "$2=correct", the distance is set to 0.

[0205] Also, if the comparison target is included in the fill-in-the-blank answer, the distance may be set to 0. For example, if the comparison target is "$2 = correct" and the fill-in-the-blank answer is "$2 = correct," the distance is set to 0.

[0206] Furthermore, if the fill-in-the-blank answer is included in the comparison target, the distance may be set to 0.5. For example, if the comparison target is "$2=correct" and the fill-in-the-blank answer is "$2=correct", the distance is set to 0.5.

[0207] In all other cases, the distance may be calculated as 1.

[0208] The hallucination presence / absence determination unit 26 may determine the presence or absence of a hallucination based on the distances between a plurality of fill-in-the-blank answers. For example, if the average value of the distances between a plurality of fill-in-the-blank answers is equal to or greater than a threshold, the hallucination presence / absence determination unit 26 may determine that the comparison target is a hallucination.

[0209] For example, as shown in Figure 22, if the comparison target is "$2 = correct" and the three fill-in-the-blank answers obtained are "$2 = correct", "$2 = correct", and "$2 = incorrect", the average value of the distances between the multiple fill-in-the-blank answers can be calculated using the following formula.

[0210] Average distance = (0 + 0 + 10) / 3 ≈ 0.33 Here, when the threshold is 0.5, the average value of the distances of the multiple fill-in-the-blank answers, 0.33, is less than the threshold of 0.5, so the hallucination presence / absence determination unit 26 determines that it is not a hallucination. The hallucination presence / absence determination unit 26 may output a hallucination presence / absence explanation indicating that it is not a hallucination as the determination result (see symbol P1 in FIG. 22).

[0211] In another example, if the comparison target is "$2 = correct" and the fill-in-the-blank answers are "$2 = incorrect", "$2 = incorrect", "$2 = incorrect", the average distance between multiple answers can be calculated using the following formula:

[0212] Average distance = (1 + 1 + 1) / 3 ≈ 1.00 Here, when the threshold is 0.5, the average value of the distances of the multiple fill-in-the-blank answers, 1.00, is greater than or equal to the threshold of 0.5, so the hallucination presence / absence determination unit 26 determines that it is a hallucination. The hallucination presence / absence determination unit 26 may output a hallucination presence / absence explanation indicating that it is a hallucination as the determination result.

[0213] The hallucination determination unit 26 determines whether the AI ​​answer is a hallucination based on the degree of agreement between the correctness determination results ($2 = correct, $2 = incorrect) included in the multiple fill-in-the-blank answers and the comparison target (standard information for correctness determination). The average distance is an example of the degree of agreement between the correctness determination results included in the multiple fill-in-the-blank answers and the comparison target.

[0214] Furthermore, the hallucination presence / absence determination unit 26 may tally up the frequency (occurrence frequency) of multiple fill-in-the-blank answers and output this as an explanatory statement together with the determination result of whether or not a hallucination occurred.

[0215] The frequency (occurrence frequency) of multiple fill-in-the-blank answers is an example of a count result of multiple answers to a fill-in-the-blank question (second sentence) output by the LLM (predetermined model).

[0216] In the example indicated by symbol P1 in Figure 22, the number of times "$2 = correct" was answered as the fill-in-the-blank answer (correct = 2 times) and the number of times "$2 = incorrect" was answered as the fill-in-the-blank answer (incorrect = 1 time) are shown.

[0217] (B) Operation The processing of the hallucination detection device 1 according to the second embodiment configured as described above will be described in accordance with the flowchart (steps C1 to C6) shown in Fig. 23, with reference to Fig. 24. Fig. 24 is a diagram for explaining the processing of the hallucination detection device 1 according to the second embodiment.

[0218] In step C1, the dialogue storage processor 22 initializes (clears) the records (rows) of the dialogue history information 32, and the blank filling processor 23 initializes (clears) the records (rows) of the blank filling history information 33. In addition, the LLM storage processor 21 registers the usage information of the LLM to be used in the LLM information 31.

[0219] These processes initialize each piece of information (LLM information 31, dialogue history information 32, and blank-filling history information 33). The dialogue-based generation AI outputs an AI answer to the user's question (see symbol D1 in FIG. 24).

[0220] In step C2, the dialogue storage processing unit 22 records (adds) the user question and the AI ​​response input via the input / output device 15 to the dialogue history information 32.

[0221] In step C3, the semantic analysis unit 27 extracts triplet pairs and sentence polarities using the AI ​​responses read from the dialogue history information 32 as input (see symbol D2 in FIG. 24).

[0222] In step C4, the fill-in-the-blank question generation unit 24 generates a fill-in-the-blank question (see reference symbol D4 in FIG. 24). The fill-in-the-blank question generation unit 24 includes a directive, a fill-in-the-blank sentence, and a hint in the fill-in-the-blank question.

[0223] At this time, the fill-in-the-blank question generation unit 24 generates two variables corresponding to the negative sentences in the AI ​​answer, for example, in the format of "$reason_serial number" and "$true / false judgment_serial number". In addition, the fill-in-the-blank question generation unit 24 adds a hint on how to answer the variable "$true / false judgment_serial number" after the instruction statement in the fill-in-the-blank question.

[0224] Furthermore, the fill-in-the-blank question generation unit 24 places a delimiter "====" after the hint in the fill-in-the-blank question, and adds a fill-in-the-blank sentence asking whether the negative sentence is true or false after this delimiter. In this fill-in-the-blank sentence, the fill-in-the-blank question generation unit 24 realizes the fill-in part by expressing the reason and true or false judgment with the variables "$reason_serial number" and "$true or false judgment_serial number" (see symbol D3 in FIG. 24).

[0225] Furthermore, the fill-in-the-blank question generating unit 24 causes the fill-in-the-blank storage processing unit 23 to register the fill-in-the-blank questions and comparison targets in the fill-in-the-blank history information 33 .

[0226] In step C5, the fill-in-the-blank answer unit 25 requests the LLM to fill in the blanks multiple times using the fill-in-the-blank questions generated by the fill-in-the-blank question generation unit 24 as prompts (see symbol D5 in Figure 24), and collects multiple answers (see symbol D6 in Figure 24).

[0227] In step C6, the hallucination determination unit 26 determines whether or not a hallucination is present based on the distance between the comparison target and the fill-in-the-blank answer. For example, the hallucination determination unit 26 compares the object (comparison target) with each fill-in-the-blank answer and calculates the distance between each fill-in-the-blank answer and the comparison target (object). The hallucination determination unit 26 then determines whether or not a hallucination is present by comparing the average value of the distances between the comparison target (object) and the multiple fill-in-the-blank answers with a threshold. The hallucination determination unit 26 outputs the determination result (a statement explaining whether or not a hallucination is present) and presents it to the user (see symbol D7 in FIG. 24 ). Thereafter, the processing ends.

[0228] (C) Effect In this way, according to the hallucination detection device 1 of the second embodiment, the semantic analysis unit 27 dynamically determines the polarity (positive or negative) of a sentence, and if the polarity of the sentence is "negative," the fill-in-the-blank question generation unit 24 creates a fill-in-the-blank question that asks whether the AI ​​answer is correct or incorrect.

[0229] In other words, if the AI ​​answer is a negative sentence, the LLM is asked to answer a fill-in-the-blank question to determine whether the entire sentence in the AI ​​answer is correct or incorrect, thereby aligning the LLM's answer to either "correct" or "incorrect," thereby aligning the direction of the answer.

[0230] This makes it possible to prevent a reversal of the judgment of whether or not a hallucination is present, which can occur, for example, when the LLM is asked to answer a fill-in-the-blank question in which the object in the sentence of the AI ​​answer is the fill-in-the-blank.

[0231] By including the variable "$reason_serial number" in the fill-in-the-blank questions that the fill-in-the-blank question generation unit 24 generates, the accuracy of fill-in-the-blank answers by the LLM can be improved. Even when a negative sentence is included in the sentence of the AI ​​answer that is the target of hallucination detection, the presence or absence of hallucination is not reversed when the LLM is asked to solve the fill-in-the-blank question in the same way as for positive sentences, with only a slight increase in the load of determining the polarity of the sentence and the load of outputting the reason for the correct / incorrect judgment. This reduces the occurrence of false positive cases. Furthermore, the scope of application of this hallucination detection method can be expanded.

[0232] This technology can suppress the decline in hallucination detection rate that can occur when the AI ​​answer is a negative sentence without creating a complex algorithm, such as changing the hallucination judgment method, thereby reducing processing and development costs.

[0233] (III) Others The disclosed technology is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the embodiments. The configurations and processes of the embodiments can be selected or combined as needed.

[0234] For example, in the first embodiment described above, the semantic analysis unit 27 determines whether the object language is Japanese or English to analyze (extract) the object language, but the present invention is not limited to this. The semantic analysis unit 27 may also determine, for example, numerical values ​​or symbols.

[0235] For example, by entering the fill-in-the-blank hint "$year_number_1," a numerical expression such as "1995" will be obtained with a high probability, which will help to avoid fill-in-the-blank answers with different directions such as "last year" or "Reiwa 1." In addition to "hypernyms" and "language," "number of characters" can also be used as a hint.

[0236] Furthermore, in the first embodiment, the fill-in-the-blank question generation unit 24 may write hints in places other than blank spaces (fill-in-the-blank places).

[0237] FIG. 25 is a diagram showing a modified example of a fill-in-the-blank question in the hallucination detection device 1 according to the first embodiment.

[0238] In the example shown in Figure 25, the variable name "$Prefecture_1" is used in place of the variable name "$Prefecture_Japanese_1" in the fill-in-the-blank sentence (see symbol P1), and the hint "$Prefecture_1 is Japanese" is also provided (see symbol P2).

[0239] Depending on the LLM, the accuracy may change depending on the position of the hint in the fill-in-the-blank question (for example, accuracy may be better if the hint is placed below the instruction), so you may change it as needed.

[0240] Furthermore, in the first embodiment, the hallucination presence / absence determination unit 26 may appropriately switch the method of calculating the distance between the object and the answer depending on the hypernym.

[0241] For example, in the case of a hypernym (such as a person, date, prefecture, or telephone number) where a difference of one character is important, the distance may be calculated as 0 when there is an exact match or when the word is included, and as 1 otherwise.

[0242] For example, in the case of hypernyms (such as organization or address) that have many similar expressions, distances can be calculated using standardized edit distance or cosine similarity between semantic vectors. A lexical comparison prompt can also be generated to have the LLM calculate the distances.

[0243] In addition to the average distance, the standard deviation of the distance may also be used to determine whether or not a hallucination occurs. The more varied the fill-in-the-blank answers, the larger the standard deviation of the distance, which is expected to improve the stability of the fill-in-the-blank answers.

[0244] In the second embodiment, the fill-in-the-blank question generation unit 24 and the hallucination determination unit 26 may also compare the reason for determining whether a question is true or false if consistency is required. In this case, "$serial number = default reason" may be recorded as the comparison target.

[0245] For example, if you compare the variable "$Reason_1" with "$1 = Because it violates company rules," you can detect whether a judgment is correct or incorrect based on a reversed reason, such as the fill-in-the-blank answer "$1 = Because it conforms to company rules."

[0246] Furthermore, to determine the distance between sentences such as reasons, particularly whether their meanings are identical or encompassing, an entailment recognition technique may be used. For example, BERT, a known general-purpose language model, may be used to convert two sentences into semantic vectors, and the cosine similarity between the semantic vectors may be calculated.

[0247] Alternatively, the LLM may be requested to perform entailment recognition between two sentences and calculate the result.

[0248] The fill-in-the-blank options for true / false judgments are not limited to "correct" or "wrong." For example, other expressions such as "supports" or "does not support," or "is supported" or "is contradictory" may be used.

[0249] Furthermore, for example, the first embodiment and the second embodiment may be combined. For example, in the polarity of the sentence extracted by the semantic analysis unit 27, if the AI ​​response is an affirmative sentence, the method according to the first embodiment may be implemented, and if the AI ​​response is a negative sentence, the method according to the second embodiment may be implemented.

[0250] Furthermore, the above disclosure enables those skilled in the art to implement and manufacture the present embodiment.

[0251] 1 Hallucination detection device 11 Processor (control unit) 12 Main memory 13 Storage device 14 Input / output interface 15 Input / output device 16 Drive device 16a Storage medium 17 Network interface 21 LLM storage processing unit 22 Dialogue storage processing unit 23 Fill-in-the-blank storage processing unit 24 Fill-in-the-blank question generation unit 25 Fill-in-the-blank answer unit 26 Hallucination presence / absence determination unit 27 Semantic analysis unit 31 LLM information 32 Dialogue history information 33 Fill-in-the-blank history information 34 Program

Claims

1. Based on a first sentence output from a predetermined model, a first word or an attribute of the first sentence contained in the first sentence is extracted. A second sentence is a query to be input into the predetermined model corresponding to the first sentence, and the second sentence is generated by embedding the input locations generated according to the attributes. Based on the multiple answers to the second sentence output by the predetermined model, it is determined whether the first sentence is a hallucination or not. A program characterized by having a computer perform a process.

2. The attribute is at least one of the superordinate terms and descriptive languages ​​of the first word contained in the first sentence, The second sentence is generated using the first word portion of the first sentence as the input location. The aforementioned generation process generates a second sentence which adds an instruction to the input location prompting the input of a response that satisfies the aforementioned attributes. The program according to feature 1.

3. The aforementioned determination process determines whether the first sentence is a hallucination by comparing the first word with a plurality of second words corresponding to the first word included in the plurality of answers. The program according to claim 1 or 2.

4. The aforementioned attribute includes information indicating whether the first sentence is a negative sentence or not. The above generation process, if the first sentence is a negative sentence, generates a second sentence using the result of the correctness judgment on the first sentence as the input location. The aforementioned generation process further generates a second sentence which includes an instruction at the input location prompting the input of a response corresponding to the correct / incorrect judgment result. The program according to feature 1.

5. The aforementioned determination process determines whether the first sentence is a hallucination or not, based on the consistency between the correct / incorrect judgment results included in the multiple answers and the criteria information for correct / incorrect judgment. The program according to feature 4.

6. In the process of making the determination, the aggregated results of multiple answers to the second sentence output by the predetermined model are output along with information indicating whether or not the first sentence is a hallucination. The program according to claim 1, characterized in that it causes the computer to perform the processing.

7. Based on a first sentence output from a predetermined model, a first word or an attribute of the first sentence contained in the first sentence is extracted. A second sentence is a query to be input into the predetermined model corresponding to the first sentence, and the second sentence is generated by embedding the input locations generated according to the attributes. Based on the multiple answers to the second sentence output by the predetermined model, it is determined whether the first sentence is a hallucination or not. A method characterized in that a computer performs the processing.

8. Based on a first sentence output from a predetermined model, a first word or an attribute of the first sentence contained in the first sentence is extracted. A second sentence is a query to be input into the predetermined model corresponding to the first sentence, and the second sentence is generated by embedding the input locations generated according to the attributes. Based on the multiple answers to the second sentence output by the predetermined model, it is determined whether the first sentence is a hallucination or not. An apparatus characterized by comprising a control unit that performs processing.