Information processing device

Abstract

It suppresses the occurrence of hallucination. [Solution] The information processing device (20) is an information processing device that determines whether or not an answer generated by a large-scale language model is incorrect, using a learning model (M) that has been trained using a user's evaluation of an answer generated by a large-scale language model and the values ​​of at least one hidden layer of the large-scale language model when it generates the answer. The information processing device comprises an acquisition means (23) for acquiring the values ​​of at least one hidden layer when the large-scale language model generates an answer to a question sentence, and a determination means (24) for determining whether or not an answer is incorrect based on the output of the learning model when the acquired values ​​are input to the learning model.

Description

Technical Field

[0001] The present invention relates to the technical field of information processing apparatuses.

Background Art

[0002] As an apparatus of this kind, for example, an apparatus has been proposed in which a query data based on a document is generated for a language model and a pair of the document and the query data is used for learning a search model for a chatbot (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] As a chatbot, a chatbot using a large language model (LLM) has been proposed. In such a chatbot, the large language model may generate a false answer. Note that the large language model is a language model constructed using a very large dataset and deep learning technology.

[0005] The present invention has been made in view of the above problems, and an object thereof is to provide an information processing apparatus capable of suppressing the occurrence of hallucination.

Means for Solving the Problems

[0006] An information processing device according to one aspect of the present invention is an information processing device that determines whether or not an answer generated by a large-scale language model is incorrect, using a learning model that has been trained using a user's evaluation of an answer generated by a large-scale language model and the values ​​of at least one hidden layer of the large-scale language model when it generates the answer, and comprises acquisition means for acquiring the values ​​of at least one hidden layer when the large-scale language model generates an answer to a question sentence, and determination means for determining whether or not an answer is incorrect based on the output of the learning model when the acquired values ​​are input to the learning model. [Brief explanation of the drawing]

[0007] [Figure 1] This is a diagram showing the configuration of an information processing system according to an embodiment. [Figure 2] This is a block diagram showing an example of a computer configuration. [Figure 3] This figure shows an example of a screen related to a chatbot. [Figure 4] This is a conceptual diagram illustrating the concept of large-scale language models. [Modes for carrying out the invention]

[0008] Embodiments of the information processing system will be described with reference to Figures 1 to 4. In Figure 1, the information processing system 1 includes a server 20. Server 20 is a server for operating a Large-Scale Language Model (LLM). For this reason, server 20 may be referred to as an LLM server. Server 20 may be a cloud server.

[0009] (Chatbot) Server 20 may provide a chatbot service. Here, Server 20 and terminal device 50 are configured to communicate with each other via a network NW. Terminal device 50 may be a personal computer, tablet device, or smartphone.

[0010] For example, user U1 may use the chatbot service via terminal device 50. In this case, user U1 may operate terminal device 50 to launch an application for using the chatbot service. User U1 may operate terminal device 50 to enter a question into the input field of the chat application. Here, "question" is not limited to interrogative sentences. For example, "question" may be a sentence that includes expressions such as requests, instructions, or commands, such as "Tell me about ****" or "Answer me about ****". Therefore, "question" is a concept that includes not only sentences in the form of interrogative sentences, but also sentences that include expressions such as requests, instructions, or commands. In other words, "question" may mean a sentence that seeks an answer from the other party.

[0011] Terminal device 50 may send the input question to server 20. Upon receiving the question, server 20 may input a prompt containing the question to the large-scale language model. Server 20 may obtain the answer to the question output from the large-scale language model. Server 20 may send the answer to terminal device 50. Upon receiving the answer, terminal device 50 may display the answer on the screen related to the chat application.

[0012] Furthermore, the chatbot service may also be a chatbot that uses a mechanism (Retrieval-Augmented Generation: RAG) that adds its own information source to the large-scale language model by combining a large-scale language model with the retrieval of a specific information source (a so-called knowledge base).

[0013] (Server 20) For example, the server 20 may be implemented by a computer COM as shown in Figure 2. In Figure 2, the computer COM comprises an arithmetic unit 110, a storage device 120, a communication device 130, an input device 140, and an output device 150. The arithmetic unit 110, storage device 120, communication device 130, input device 140, and output device 150 are connected via a data bus 160.

[0014] The arithmetic unit 110 may have a processor. Furthermore, the arithmetic unit 110 may have a single processor or multiple processors. In other words, the arithmetic unit 110 may have one or more processors. Furthermore, the processor may be a multi-core processor. If the arithmetic unit 11 has a single processor that is a multi-core processor, then logically, the arithmetic unit 110 can be said to have multiple processors.

[0015] The processor may be at least one of the following: CPU (Central Processing Unit), GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), and TPU (Tensor Processing Unit).

[0016] The storage device 120 may be at least one of the following: RAM (Random Access Memory), ROM (Read Only Memory), hard disk drive, magneto-optical disk drive, SSD (Solid State Drive), and optical disk array. In other words, the storage device 120 may be implemented by a single device or by multiple devices.

[0017] The communication device 130 may be capable of communicating with devices outside of the computer COM. Furthermore, the communication device 130 may use either wired or wireless communication.

[0018] The input device 140 is a device capable of receiving input of information to the computer COM from the outside. The input device 140 may include an operating device (e.g., keyboard, mouse, touch panel, etc.) operable by the user of the computer COM. The input device 140 may include a recording medium reader capable of reading information recorded on a removable recording medium for the computer COM, such as a USB (Universal Serial Bus) memory. Incidentally, when information is input to the computer COM via the communication device 130 (in other words, when the computer COM acquires information via the communication device 130), the communication device 130 may function as an input device.

[0019] The output device 150 is a device capable of outputting information to the outside of the computer COM. The output device 150 may have a display device capable of outputting visual information such as characters and images as the above information. The output device 150 may also have a speaker capable of outputting auditory information such as sound as the above information. The output device 150 may also have a vibration motor capable of outputting tactile information such as vibration as the above information. The output device 150 may have a printer. The output device 150 may be capable of outputting information to a removable recording medium for the computer COM, such as a USB memory. Incidentally, when the computer COM outputs information via the communication device 130, the communication device 130 may function as an output device.

[0020] The storage device 120 is capable of storing desired data. The storage device 120 may store the computer program CP executed by the arithmetic unit 110. The storage device 120 may temporarily store data temporarily used by the arithmetic unit 110 when the arithmetic unit 110 is executing the computer program CP.

[0021] Furthermore, the computer program CP may be computer-readable and recorded on a non-transitory recording medium. In this case, the computer program CP may be stored in the storage device 120 by reading the recording medium using a recording medium reader (not shown) provided in the computer COM. As the recording medium, at least one of an optical disk, a magnetic medium, a magneto-optical disk, a semiconductor memory, and any other medium capable of storing a program may be used. Furthermore, the computer program CP may be acquired from a device (not shown) outside the computer COM via the communication device 130. In other words, the computer program CP may be downloaded from an external device to the storage device 120 of the computer COM.

[0022] The arithmetic unit 110 (e.g., a processor) may execute the processing to be performed by the computer COM together with the storage device 120 in which the computer program CP is stored (in other words, together with the storage device 120 and the computer program CP stored in the storage device 120). For example, by executing the computer program CP, a logical functional block for executing the processing to be performed by the computer COM may be realized in the arithmetic unit 110 (e.g., within the processor).

[0023] As shown in FIG. 1, the server 20 includes an input unit 21, an output unit 22, an acquisition unit 23, and a determination unit 24. The input unit 21, the output unit 22, the acquisition unit 23, and the determination unit 24 may be realized as the above-described logical functional blocks. Furthermore, at least one of the input unit 21, the output unit 22, the acquisition unit 23, and the determination unit 24 may be realized as a physical processing circuit. At least one of the input unit 21, the output unit 22, the acquisition unit 23, and the determination unit 24 may be realized in a form in which logical functional blocks and physical processing circuits are mixed. As shown in FIG. 1, the determination unit 24 has a model M.

[0024] (Model M) For example, Model M may be a model constructed using machine learning (i.e., a trained model). An example of how to construct Model M will be explained with reference to Figures 3 and 4.

[0025] As described above, in a chatbot service, terminal device 50 that receives a response from a large-scale language model may display the response on a screen related to the chat application. In this case, terminal device 50 may display screen 51 as shown in Figure 3. For example, screen 51 may include an area 511 that displays the question entered by user U1 and an area 512 that displays the response from the large-scale language model. For example, below area 512, buttons 513 and 513 may be placed for user U1 to input their evaluation of the response from the large-scale language model. In the example shown in Figure 3, user U1's evaluation is a two-level evaluation of "GOOD" and "BAD". However, user U1's evaluation is not limited to a two-level evaluation, but may be a three-level or higher evaluation.

[0026] User U1 may operate the terminal device 50 to select either button 513 or 514 (in other words, User U1 may evaluate the response of the large-scale language model). If User U1 selects either button 513 or 514, the terminal device 50 sends User U1's evaluation to the server 20.

[0027] For example, as shown in Figure 4, the large-scale language model has an input layer, an output layer, and multiple hidden layers. Note that "hidden layers" may also be referred to as "hidden layers." Server 20 retrieves the value of at least one hidden layer (e.g., hidden layer MLx) of the large-scale language model when it generates the answer evaluated by user U1. Therefore, the timing of retrieving the value of at least one hidden layer is before the timing of user U1 evaluating the answer.

[0028] Now, answers that user U1 gives a high rating to (e.g., "GOOD") are likely to be appropriate answers to user U1's question. On the other hand, answers that user U1 gives a low rating to (e.g., "BAD") are likely to be inappropriate answers to user U1's question. In other words, answers that user U1 gives a low rating to (e.g., "BAD") are likely to be incorrect. In addition, the values ​​of the intermediate layer of the large-scale language model are likely to differ depending on whether the large-scale language model generates an appropriate answer or an inappropriate answer.

[0029] In this embodiment, machine learning of model M may be performed using the values ​​of at least one intermediate layer (e.g., intermediate layer MLx) of a large-scale language model and the evaluation of a user (e.g., user U1) as training data. In this case, supervised learning may be performed using the values ​​of at least one intermediate layer of the large-scale language model as input data and the user's evaluation as ground truth data. For example, model M may be trained to output that it is an appropriate answer when the input value of at least one intermediate layer corresponds to a high user evaluation, and to output that it is an inappropriate answer when the input value of at least one intermediate layer corresponds to a low user evaluation. For example, model M may be constructed by such machine learning.

[0030] (Server 20 operation) Returning to Figure 1, let's explain the operation of the server 20. The server 20 and the terminal device 60 are configured to communicate with each other via a network NW. The terminal device 60 may be a personal computer, a tablet terminal, or a smartphone.

[0031] For example, user U2 may use the chatbot service via terminal device 60. In this case, user U2 may operate terminal device 60 to launch an application for using the chatbot service. User U2 may operate terminal device 60 to enter a question into the input field of the chat application. Terminal device 60 sends the question to server 20 via the network NW.

[0032] Upon receiving a question, the input unit 21 of the server 20 inputs a prompt containing the question to the large-scale language model. The large-scale language model generates an answer to the question and outputs it to the output unit 22. At this time, the acquisition unit 23 acquires the value of at least one intermediate layer (e.g., intermediate layer MLx) of the large-scale language model. The acquisition unit 23 outputs the acquired value to the determination unit 24. The determination unit 24 inputs the value acquired by the acquisition unit 23 to the model M. Based on the output of the model M, the determination unit 24 determines whether the above answer is incorrect or not.

[0033] If the first answer is determined not to be incorrect, the determination unit 25 permits the output unit 23 to transmit (in other words, output) the first answer to the terminal device 60. On the other hand, if the first answer is determined to be incorrect, the determination unit 24 prohibits the output unit 22 from transmitting the first answer to the terminal device 60. In this case, the output unit 22 may transmit information to the terminal device 60 to cause it to output a message such as "No suitable answer was found."

[0034] (Technical effects) In chatbot services, large-scale language models can sometimes generate false answers (in other words, content that is not factual). This means that hallucination can occur in chatbot services. In the information processing system 1 according to this embodiment, before the answer generated by the large-scale language model is sent to a terminal device (for example, terminal device 60), the determination unit 24 of the server 20 determines whether the answer is incorrect or not. If the answer is determined to be incorrect, the transmission of the answer to the terminal device is prohibited. In other words, in the information processing system 1, although the large-scale language model may generate incorrect answers, it is possible to suppress the presentation of incorrect answers to the user. Therefore, according to the information processing system 1, the occurrence of hallucination can be suppressed.

[0035] Various aspects of the invention derived from the embodiments described above are described below.

[0036] An information processing device according to one aspect of the invention is an information processing device that determines whether or not an answer generated by a large-scale language model is incorrect, using a learning model that has been trained using a user's evaluation of an answer generated by a large-scale language model and the values ​​of at least one hidden layer of the large-scale language model when it generates the answer, and comprises acquisition means for acquiring the values ​​of at least one hidden layer when the large-scale language model generates an answer to a question sentence, and determination means for determining whether or not an answer is incorrect based on the output of the learning model when the acquired values ​​are input to the learning model.

[0037] In the above-described embodiment, "server 20" corresponds to an example of an "information processing device," "acquisition unit 23" corresponds to an example of an "acquisition means," "determination unit 25" corresponds to an example of a "determination means," and "model M" corresponds to an example of a "learning model."

[0038] In the information processing device according to the above embodiment, if it is determined that the first answer is incorrect, the determination means may prohibit the output of the first answer. By configuring it in this way, it is possible to prevent an incorrect answer from being presented to the user.

[0039] The present invention is not limited to the embodiments described above, and can be modified as appropriate without contradicting the gist or idea of ​​the invention as can be read from the claims and specification as a whole. Information processing devices that involve such modifications are also included within the technical scope of the present invention. [Explanation of Symbols]

[0040] 1... Information processing system, 20... Server, 23... Acquisition unit, 24... Determination unit

Claims

1. An information processing device that determines whether or not the answer generated by a large-scale language model is erroneous, using a learning model trained with a user's evaluation of the answer generated by the large-scale language model and the values ​​of at least one hidden layer of the large-scale language model when generating the answer, A means for obtaining the value of at least one hidden layer when the large-scale language model generates an answer to a question sentence, A determination means for determining whether the first answer is incorrect based on the output of the learning model when the acquired value is input to the learning model, An information processing device equipped with the following features.

2. If the above answer is determined to be incorrect, the determination means prohibits the output of the above answer. The information processing apparatus according to claim 1.

Images