Information processing apparatus

Abstract

An object of the present application is to suppress the generation of hallucinations. An information processing apparatus determines whether an answer generated by a large-scale language model is incorrect using a learning model that has been learned using a user evaluation of the answer generated by the large-scale language model and values of at least one hidden layer of the large-scale language model at the time of generation of the answer. The information processing apparatus includes an acquisition unit that acquires values of at least one hidden layer at the time of generation of one answer to one question by the large-scale language model, and a determination unit that determines whether the one answer is incorrect based on an output of the learning model when the acquired values are input to the learning model.

Description

Technical Field

[0001] This invention relates to the technical field of information processing devices. Background Technology

[0002] As such a device, for example, a device has been proposed that enables a language model to generate document-based query data and uses the pairing of documents and query data for learning the retrieval model of a chatbot (see Patent Document 1).

[0003] Patent Document 1: Japanese Patent Application Publication No. 2023-076413 Summary of the Invention

[0004] As a chatbot, a chatbot using Large Language Models (LLM) has been proposed. In such chatbots, LLMs may generate false responses. Furthermore, LLMs refer to language models built using extremely large datasets and deep learning techniques.

[0005] The present invention was made in view of the above-mentioned problems, and its objective is to provide an information processing device capable of suppressing the generation of hallucinations.

[0006] One aspect of the present invention relates to an information processing apparatus that uses a learning model to determine whether an answer generated by a large-scale language model is incorrect. The learning model is a model learned using user ratings of the answers generated by the large-scale language model and values ​​of at least one hidden layer of the large-scale language model at the time the answer was generated. The information processing apparatus comprises: an acquisition unit that acquires the values ​​of the at least one hidden layer when the large-scale language model generates an answer to a question; and a determination unit that determines whether the answer is incorrect based on the output of the learning model when the acquired values ​​are input into the learning model. Attached Figure Description

[0007] Figure 1 This is a diagram showing the structure of the information processing system involved in the implementation method.

[0008] Figure 2 This is a block diagram illustrating an example of the structure of a computer.

[0009] Figure 3 This is an example of a screen involved in a chatbot.

[0010] Figure 4 It is a concept graph representing a large-scale language model. Detailed Implementation

[0011] refer to Figures 1 to 4 The implementation methods involved in the information processing system are described. Figure 1 In this system, information processing system 1 includes server 20. Server 20 is a server used for applying Large Scale Language Models (LLM). Therefore, server 20 can be called an LLM server. Alternatively, server 20 can be a cloud server.

[0012] (Chatbot)

[0013] Server 20 can provide chatbot services. Here, server 20 and terminal device 50 are configured to communicate with each other via a network NW. Furthermore, terminal device 50 can be a personal computer, tablet, or smartphone.

[0014] For example, user U1 can utilize the chatbot service via terminal device 50. In this case, user U1 can operate terminal device 50 to launch an application for utilizing the chatbot service. User U1 can operate terminal device 50 to enter a question in the input field of the chat application. Here, "question" is not limited to interrogative sentences. For example, "question" can be a sentence containing expressions such as "Please tell me about ****" or "Please answer about ****". Therefore, "question" is not limited to sentences in the form of questions, but includes the concept of sentences containing expressions such as requests, instructions, and commands. That is, "question" can refer to a sentence seeking an answer from the other party.

[0015] Terminal device 50 can send the input query to server 20. Server 20, upon receiving the query, can input a prompt containing the query into a large-scale language model. Server 20 can obtain the answer to the query from the large-scale language model. Server 20 can send the answer to terminal device 50. Terminal device 50, upon receiving the answer, can display the answer on the screen displayed in the chat application.

[0016] Alternatively, a chatbot service can be a chatbot that uses a mechanism that gives a large-scale language model a unique information source by combining it with retrieval from a specific information source (so-called a knowledge base) (Retrieval-Augmented Generation: RAG).

[0017] (Server 20)

[0018] For example, server 20 can be... Figure 2 The computer COM implementation shown. Figure 2In this computer COM, there are arithmetic unit 110, storage unit 120, communication unit 130, input unit 140 and output unit 150. The arithmetic unit 110, storage unit 120, communication unit 130, input unit 140 and output unit 150 are connected via data bus 160.

[0019] The arithmetic unit 110 may have a processor. Furthermore, the arithmetic unit 110 may have a single processor or multiple processors. That is, the arithmetic unit 110 may have more than one processor. Additionally, the processor may be a multi-core processor. In the case where the arithmetic unit 110 has a single processor that functions as a multi-core processor, it can be said that the arithmetic unit 110 logically has multiple processors.

[0020] The processor may be at least one of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a Field Programmable Gate Array (FPGA), and a Tensor Processing Unit (TPU).

[0021] Storage device 120 may be at least one of random access memory (RAM), read-only memory (ROM), hard disk drive, magneto-optical disk drive, solid-state drive (SSD), and optical disk array. That is, storage device 120 may be implemented by a single device or by multiple devices.

[0022] The communication device 130 can communicate with external devices of the computer COM. Furthermore, the communication device 130 can perform both wired and wireless communication.

[0023] Input device 140 is a device capable of accepting information input from an external source to the computer COM. Input device 140 may include user-operable devices of the computer COM (e.g., keyboard, mouse, touch panel, etc.). Input device 140 may, for example, include a recording medium reading device capable of reading information recorded on a recording medium removable from the computer COM, such as a Universal Serial Bus (USB) memory. Furthermore, when information is input to the computer COM via communication device 130 (in other words, when the computer COM obtains information via communication device 130), communication device 130 may function as an input device.

[0024] Output device 150 is a device capable of outputting information to an external computer COM port. Output device 150 may include a display device capable of outputting visual information such as characters and images as the information. Additionally, output device 150 may include a speaker capable of outputting auditory information such as sound as the information. Output device 150 may include a vibration motor capable of outputting tactile information such as vibration as the information. Output device 150 may include a printer. Output device 150 may, for example, output information to a recording medium removable from the computer COM port, such as a USB memory. Furthermore, when the computer COM port outputs information via communication device 130, communication device 130 may function as an output device.

[0025] Storage device 120 is capable of storing desired data. The computer program CP executed by the arithmetic unit 110 can be stored in storage device 120. When the arithmetic unit 110 executes the computer program CP, storage device 120 can temporarily store data temporarily used by the arithmetic unit 110.

[0026] Furthermore, the computer program CP can be recorded on a computer-readable and non-temporary recording medium. In this case, the computer program CP can be stored in the storage device 120 by reading the recording medium using a recording medium reading device (not shown) provided with the computer COM. Additionally, at least one of optical discs, magnetic media, magneto-optical discs, semiconductor memory, and any medium capable of storing other programs can be used as the recording medium. Furthermore, the computer program CP can also be obtained from an external (not shown) device of the computer COM via the communication device 130. In other words, the computer program CP can also be downloaded from an external device to the storage device 120 of the computer COM.

[0027] The arithmetic unit 110 (e.g., a processor) can perform the processing that the computer COM should perform together with the storage device 120 storing the computer program CP (in other words, together with the storage device 120 and the computer program CP stored in the storage device 120). For example, by executing the computer program CP, the arithmetic unit 110 can implement logical function blocks for performing the processing that the computer COM should perform within the arithmetic unit 110 (e.g., within the processor).

[0028] like Figure 1As shown, server 20 includes an input unit 21, an output unit 22, an acquisition unit 23, and a determination unit 24. The input unit 21, output unit 22, acquisition unit 23, and determination unit 24 can be implemented as the aforementioned logic function blocks. Alternatively, at least one of the input unit 21, output unit 22, acquisition unit 23, and determination unit 24 can be implemented as a physical processing circuit. At least one of the input unit 21, output unit 22, acquisition unit 23, and determination unit 24 can be implemented in a manner that combines logic function blocks and physical processing circuits. Figure 1 As shown, the decision unit 24 has a model M.

[0029] (Model M)

[0030] For example, model M can be a model built through machine learning (i.e., a learned model). (See reference) Figure 3 and Figure 4 An example of how to construct model M will be given.

[0031] As described above, in a chatbot service, the terminal device 50 that receives a response from a large-scale language model can display the response on the screen involved in the chat application. In this case, it can be displayed on the terminal device 50. Figure 3 The screen 51 shown. For example, screen 51 may include an area 511 displaying the question entered by user U1 and an area 512 displaying the answer from the large-scale language model. For example, buttons 513 and 514 for user U1 to input their evaluation of the large-scale language model's answer may be configured below area 512. Figure 3 In the example shown, user U1's rating is a two-stage rating of "good" and "bad". However, user U1's rating is not limited to two stages; it can be a rating of three or more stages.

[0032] User U1 can operate terminal device 50 to select button 513 or 514 (in other words, user U1 can evaluate the answer of the large-scale language model). If user U1 selects button 513 or 514, terminal device 50 sends user U1's evaluation to server 20.

[0033] For example, such as Figure 4 As shown, the large-scale language model has an input layer, an output layer, and multiple intermediate layers. Furthermore, the "intermediate layers" can also be called "hidden layers." Server 20 obtains the values ​​of at least one intermediate layer (e.g., intermediate layer MLx) of the large-scale language model when the large-scale language model generates an answer evaluated by user U1. Therefore, the time when the values ​​of at least one intermediate layer are obtained is before the time when user U1 evaluates the answer.

[0034] However, a highly rated (e.g., "good") answer from user U1 is highly likely to be an appropriate answer to user U1's question. Conversely, a low-rated (e.g., "bad") answer from user U1 is highly likely to be an inappropriate answer to user U1's question. In other words, a low-rated (e.g., "bad") answer from user U1 is highly likely to be incorrect. Furthermore, large-scale language models are highly likely to have different values ​​for their intermediate layers when generating appropriate and inappropriate answers.

[0035] In this embodiment, machine learning of model M can be performed using the values ​​of at least one intermediate layer (e.g., intermediate layer MLx) of a large-scale language model and user ratings (e.g., user U1) as learning data. In this case, supervised learning can be performed using the values ​​of at least one intermediate layer of the large-scale language model as input data and the user ratings as correct data. For example, model M can learn to output an appropriate response when the input is a value of at least one intermediate layer with a high user rating, and an inappropriate response when the input is a value of at least one intermediate layer with a low user rating. Model M can be constructed using this type of machine learning.

[0036] (Action of server 20)

[0037] return Figure 1 The operation of server 20 will be described below. Server 20 and terminal device 60 are configured to communicate with each other via network NW. Terminal device 60 can be a personal computer, tablet computer, or smartphone.

[0038] For example, user U2 can utilize the chatbot service via terminal device 60. In this case, user U2 can operate terminal device 60 to launch an application for utilizing the chatbot service. User U2 can operate terminal device 60 to enter a question in the input field of the chat application. Terminal device 60 sends a question to server 20 via network NW.

[0039] Upon receiving a question, the input unit 21 of server 20 inputs a prompt containing the question into a large-scale language model. The large-scale language model generates an answer to the question and outputs it to output unit 22. At this time, acquisition unit 23 acquires the value of at least one intermediate layer (e.g., intermediate layer MLx) of the large-scale language model. Acquisition unit 23 outputs the acquired value to decision unit 24. Decision unit 24 inputs the value acquired by acquisition unit 23 into model M. Based on the output of model M, decision unit 24 determines whether the above answer is incorrect.

[0040] If a response is determined to be correct, the determination unit 25 allows the output unit 23 to send (in other words, output) a response to the terminal device 60. Conversely, if a response is determined to be incorrect, the determination unit 24 prevents the output unit 22 from sending a response to the terminal device 60. In this case, the output unit 22 may send information to the terminal device 60 to cause it to output a message such as "No appropriate response found".

[0041] (Technical effect)

[0042] In chatbot services, large-scale language models sometimes generate false answers (in other words, content that differs from the facts). That is, illusions can sometimes occur in chatbot services. In the information processing system 1 according to this embodiment, before the answer generated by the large-scale language model is sent to the terminal device (e.g., terminal device 60), the determination unit 24 of the server 20 determines whether the answer is incorrect. Then, if the answer is determined to be incorrect, sending the answer to the terminal device is prohibited. That is, in the information processing system 1, although the large-scale language model sometimes generates incorrect answers, it is possible to suppress the presentation of incorrect answers to the user. Therefore, according to the information processing system 1, the generation of illusions can be suppressed.

[0043] Hereinafter, various aspects of the invention derived from the embodiments described above will be described.

[0044] One aspect of the invention relates to an information processing apparatus that uses a learning model to determine whether an answer generated by a large-scale language model is incorrect. The learning model is a model learned using user ratings of the answers generated by the large-scale language model and values ​​of at least one hidden layer of the large-scale language model at the time the answer was generated. The information processing apparatus includes: an acquisition unit that acquires the values ​​of the at least one hidden layer when the large-scale language model generates an answer to a question; and a determination unit that determines whether the answer is incorrect based on the output of the learning model when the acquired values ​​are input into the learning model.

[0045] In the above embodiments, "server 20" is equivalent to an example of "information processing device", "acquisition unit 23" is equivalent to an example of "acquisition unit", "determination unit 25" is equivalent to an example of "determination unit", and "model M" is equivalent to an example of "learning model".

[0046] In the information processing apparatus described above, if an answer is determined to be incorrect, the determination unit can prohibit the output of that answer. If configured in this way, it is possible to prevent incorrect answers from being presented to the user.

[0047] The present invention is not limited to the embodiments described above, and appropriate modifications can be made without departing from the spirit or concept of the invention as read in its entirety from the claims and description. Information processing apparatuses that accompany such modifications are also included within the technical scope of the present invention.

[0048] Symbol Explanation

[0049] 1-Information processing system, 20-Server, 23-Acquisition unit, 24-Judgment unit.

Claims

1. An information processing device, characterized in that, The information processing device uses a learning model to determine whether the answer generated by the large-scale language model is incorrect. The learning model is a model that has been trained using user ratings of the answers generated by the large-scale language model and the values ​​of at least one hidden layer of the large-scale language model at the time the answer was generated. The information processing device includes: The acquisition unit acquires the values ​​of at least one hidden layer when the large-scale language model generates an answer to a question; and The determination unit determines whether an answer is incorrect based on the output of the learning model when the acquired value is input into the learning model.

2. The information processing device according to claim 1, characterized in that, If an answer is determined to be incorrect, the determination unit prohibits the output of that answer.

Images