Terminal devices, information processing methods, programs, and systems

Abstract

In processing using a language model, confidential information is handled solely by the terminal device, and the processing load on the terminal device is reduced. [Solution] The terminal device includes at least: a transmitting unit that transmits question information representing a question entered by a user using an input device to an information processing device having a first language model that answers the question; a receiving unit that receives first answer information obtained by inputting a first prompt generated using the question information and general information related to the question information into the first language model from the information processing device; a prompt generating unit that generates a second prompt using the question information, the first answer information and pre-set secret information related to the question information; and an answer generating unit that inputs the second prompt into a second language model to generate second answer information.

Description

Technical Field

[0001] The present disclosure relates to a terminal device, an information processing method, a program, and a system that utilize a language model.

Background Art

[0002] Conventionally, in a system that uses a large language model to obtain answers to questions, answers are obtained using secret information (e.g., private information, confidential information, etc.) stored in the cloud. However, as a user, since there is a risk of leakage, it is desirable to avoid placing secret information on the cloud as much as possible.

[0003] Therefore, it is conceivable to perform processing using a large language model on the terminal device side. However, as the processing volume of the terminal device increases, the terminal device operates at a low speed, so the time until an answer is obtained may increase. Also, when the operating speed of the terminal device is improved and the accuracy of the answer is made comparable to that of the cloud, the hardware size of the terminal device becomes large. On the other hand, when the hardware size of the terminal device is kept as it is and the operating speed is improved, the accuracy of the answer decreases.

[0004] As a related technique, Patent Document 1 discloses a terminal device that performs highly accurate character recognition processing at low cost and reduces the risk of leakage of secret information. According to the terminal device of Patent Document 1, based on a document image including personal information, the document image is separated into a personal information image including personal information and a general information image not including personal information, and the general information image is transmitted to a cloud server. Further, the terminal device receives a general recognition result, which is the recognition result of character recognition processing for the general information image, from the cloud server, and obtains the recognition result of character recognition processing for the document image according to the general recognition result and the information based on the personal information image.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

[0006] However, in the terminal device described in Patent Document 1, the processed image containing personal information separated from the document image is transmitted to the character recognition processing device, which means that it may be deciphered on the cloud side (potentially leading to the leakage of confidential information).

[0007] One example of the purpose of this disclosure is to handle confidential information solely on the terminal device side and to reduce the processing load on the terminal device in processing that utilizes a language model. [Means for solving the problem]

[0008] To achieve the above objective, the terminal device in one aspect of this disclosure is At a minimum, a transmitting unit that transmits question information representing a question entered by a user using an input device to an information processing device having a first language model that answers the question, A receiving unit receives first response information obtained by inputting a first prompt, generated using the aforementioned question information and general information related to the aforementioned question information, into the first language model, from the information processing device. A prompt generation unit generates a second prompt using the aforementioned question information, the first answer information, and pre-set secret information related to the aforementioned question information. A response generation unit that inputs the second prompt to a second language model to generate second response information, It is characterized by having the following features.

[0009] Furthermore, in order to achieve the above objectives, the information processing method in one aspect of this disclosure is: Computers At a minimum, question information representing a question entered by a user using an input device is transmitted to an information processing device having a first language model that answers the question. The first response information obtained by inputting a first prompt, generated using the aforementioned question information and general information related to the aforementioned question information, into the first language model is received from the information processing device. A second prompt is generated using the aforementioned question information, the first answer information, and the pre-configured secret information related to the aforementioned question information. The second prompt is input to the second language model to generate the second response information. It is characterized by the following:

[0010] Furthermore, in order to achieve the above objectives, the program in one aspect of this disclosure is On the computer, At a minimum, the system transmits question information representing a question entered by a user using an input device to an information processing device having a first language model that answers the question. The first response information obtained by inputting a first prompt, generated using the aforementioned question information and general information related to the aforementioned question information, into the first language model is received from the information processing device. Using the aforementioned question information, the first answer information, and the pre-configured secret information related to the aforementioned question information, a second prompt is generated. The second prompt is input to the second language model to generate the second response information. It is characterized by the following:

[0011] Furthermore, in order to achieve the above objectives, the system in one aspect of this disclosure is The terminal device transmits question information, which represents a question entered by the user using an input device, to a server device having a first language model that answers the question. The aforementioned server device The terminal device receives the question information, A first prompt is generated using the aforementioned question information and general information related to the aforementioned question information. The first prompt is input to the first language model to generate the first response information. Transmit the first response information to the terminal device, The terminal device Receives the first response information from the server device, Generates a second prompt using the question information, the first response information, and secret information related to the question information set in advance, Inputs the second prompt into a second language model to generate second response information. It is characterized by this.

[0012] To achieve the above object, a terminal device in one aspect of the present disclosure At least, a transmission unit that transmits question information representing a question input by a user using an input device to an information processing device having a first language model that answers the question, When generating a first response information by inputting a first prompt generated using the question information and general information related to the question information into the first language model, cache information read from a memory used in the first language model, and a receiving unit that receives the cache information and the general information from the information processing device, A writing unit that writes the cache information into a memory used in a second language model, A prompt generation unit that generates a second prompt using the question information, the general information, and secret information related to the question information set in advance, [[ID=2३]] An answer generation unit that inputs the second prompt into the second language model to generate second response information, It is characterized by having.

[0013] Also, to achieve the above object, an information processing method in one aspect of the present disclosure A computer At least transmits question information representing a question input by a user using an input device to an information processing device having a first language model that answers the question, When generating the first response information by inputting the first prompt generated using the question information and general information related to the question information into the first language model, cache information is read from the memory used in the first language model, and the cache information and the general information are received from the information processing device. Writing means for writing the cache information into the memory used in the second language model. Generate a second prompt using the question information, the general information, and secret information related to the question information set in advance. Input the second prompt into the second language model to generate second response information. It is characterized by the above.

[0014] Also, to achieve the above object, a program in one aspect of the present disclosure causes a computer to at least transmit question information representing a question input by a user using an input device to an information processing device having a first language model that answers the question, When generating the first response information by inputting the first prompt generated using the question information and general information related to the question information into the first language model, cache information is read from the memory used in the first language model, and the information processing device is caused to receive the cache information and the general information. Write the cache information into the memory used in the second language model. Generate a second prompt using the question information, the general information, and secret information related to the question information set in advance. Input the second prompt into the second language model to generate second response information. It is characterized by the above. It is characterized by the above.

[0015] Furthermore, to achieve the above object, a system in one aspect of the present disclosure The terminal device transmits question information, which represents a question entered by the user using an input device, to a server device having a first language model that answers the question. The aforementioned server device The terminal device receives the question information, A first prompt is generated using the aforementioned question information and general information related to the aforementioned question information. The first prompt is input to the first language model to generate the first response information. When generating the first response information, the cache information used by the first language model is read from the memory, The cache information and the general information are transmitted to the terminal device. The terminal device, The server device receives the cache information and the general information, The write operation involves writing the aforementioned cache information to the memory used by the second language model. A second prompt is generated using the aforementioned question information, the aforementioned general information, and the pre-configured secret information related to the aforementioned question information. The second prompt is input to the second language model to generate the second response information. It is characterized by the following: [Effects of the Invention]

[0016] As described above, according to this disclosure, in processing using a language model, confidential information can be handled solely on the terminal device side, and the processing load on the terminal device can be reduced. [Brief explanation of the drawing]

[0017] [Figure 1] Figure 1 is a diagram illustrating an example of a terminal device according to Embodiment 1. [Figure 2] Figure 2 is a diagram illustrating an example of a system having a terminal device according to Embodiment 1. [Figure 3] Figure 3 is a diagram illustrating an example of the operation of the terminal device of Embodiment 1. [Figure 4] Figure 4 is a diagram illustrating an example of the operation of the information processing device (server device) of Embodiment 1. [Figure 5] Figure 5 is a diagram illustrating an example of the operation of the system of Embodiment 1. [Figure 6] Figure 6 is a diagram illustrating an example of a terminal device according to Embodiment 2. [Figure 7] Figure 7 is a diagram illustrating an example of a system having a terminal device according to Embodiment 2. [Figure 8] Figure 8 is a diagram illustrating an example of the operation of the terminal device of Embodiment 2. [Figure 9] Figure 9 is a diagram illustrating an example of the operation of the information processing device (server device) of Embodiment 2. [Figure 10] Figure 10 is a diagram illustrating an example of the operation of the system of Embodiment 2. [Figure 11] Figure 11 is a diagram illustrating an example of a computer that implements the terminal device and information processing device (server device) in Embodiments 1 and 2. [Modes for carrying out the invention]

[0018] Embodiments will be described below with reference to the drawings. In the drawings described below, elements having the same function or corresponding function will be denoted by the same reference numeral, and repeated descriptions of such elements may be omitted.

[0019] (Embodiment 1) The configuration of the terminal device in Embodiment 1 will be explained using Figure 1. Figure 1 is a diagram illustrating an example of the terminal device in Embodiment 1.

[0020] [Device configuration] The terminal device shown in Figure 1 is a device that, in processing using a language model, handles confidential information solely on the terminal device side and reduces the processing load on the terminal device. Also, as shown in Figure 1, the terminal device 10 has a transmission unit 11, a reception unit 12, a prompt generation unit 13, and a response generation unit 14.

[0021] The transmission unit 11 transmits at least question information representing a question entered by a user using an input device to an information processing device (server device) having a first language model that outputs an answer to the question.

[0022] The receiving unit 12 receives first response information from the information processing device, which is obtained by inputting a first prompt, generated using the question information and general information related to the question information, into the first language model.

[0023] The prompt generation unit 13 generates a second prompt using the question information, the first answer information, and the secret information related to the pre-configured question information.

[0024] The response generation unit 14 inputs the second prompt to the second language model to generate the second response information.

[0025] Thus, in Embodiment 1, since confidential information is handled by the terminal device without being transmitted to the cloud side (information processing device (server device)) during processing using the language model, leakage of confidential information can be suppressed.

[0026] Furthermore, since the cloud first obtains the first answer information using the question information and general information, and the terminal device obtains the second answer information using the question information, the first answer information, and confidential information, the processing load on the language model on the terminal device can be reduced.

[0027] [System Configuration] Next, the configuration of the terminal device 10 in Embodiment 1 will be described in more detail using Figure 2. Figure 2 is a diagram illustrating an example of a system having the terminal device of Embodiment 1.

[0028] As shown in Figure 2, the system 100 in Embodiment 1 includes a terminal device 10, an information processing device 20 (server device), and a network 30.

[0029] The terminal device 10 is, for example, a programmable device such as a CPU (Central Processing Unit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or a circuit equipped with one or more of these, an information processing device located on the edge side of a server computer, personal computer, mobile terminal, etc.

[0030] The information processing device 20 (server device) is, for example, a CPU, a programmable device such as an FPGA, a GPU, or a circuit equipped with one or more of these, or an information processing device located on the cloud side such as a server computer.

[0031] Network 30 is a general network constructed using communication lines such as the Internet, LAN (Local Area Network), dedicated lines, telephone lines, corporate networks, mobile communication networks, Bluetooth (registered trademark), and Wi-Fi (Wireless Fidelity) (registered trademark).

[0032] ●The terminal device will be explained in detail. The terminal device 10 includes an operation unit 15, a transmission unit 11, a reception unit 12, a prompt generation unit 13, an answer generation unit 14, a language model 16, and a storage unit 17.

[0033] The operation unit 15 is a user interface for inputting questions using input devices such as a touch panel, mouse, or keyboard, which are provided on the terminal device 10. However, the operation unit 15 may be provided outside the terminal device 10.

[0034] First, the transmission unit 11 obtains question information (R: request) representing a question entered by the user using at least the operation unit 15. Next, the transmission unit 11 transmits this information to the information processing device 20 (server device), which has a language model 25 (first language model: LLM1) that answers the question, as will be described later.

[0035] The receiving unit 12 receives the first response information (A1=LLM1(prompt1)) from the information processing device 20 by inputting the first prompt (prompt1=G+R), which is generated using the question information (R) and general information (G:general) related to the question information (R), into the language model 25 (LLM1).

[0036] General information (G) is information that is pre-stored in the memory unit 26, which will be described later. General information (G) is, for example, information that would not cause problems even if leaked to the outside.

[0037] The prompt generation unit 13 generates a second prompt (prompt2=R+A1+P) using the question information (R), the first answer information (A1), and the secret information (P:private) associated with the pre-configured question information. Confidential information (P) is information that is pre-stored in the memory unit 17. Confidential information (P) is, for example, information that must not be leaked to the outside.

[0038] The response generation unit 14 inputs the second prompt (prompt2) into the language model 16 (second language model: LLM2) to generate the second response information (A2=LLM2(prompt2)).

[0039] Language Model 16 (LLM2) is, for example, a large-scale language model. Examples of large-scale language models include self-attention-based large-scale language models. However, it is not limited to the large-scale language models mentioned above.

[0040] The configuration of language model 16 is not limited; it may be equivalent to or different from language model 25.

[0041] The memory unit 17 is a circuit with memory. The memory unit 17 stores information such as, for example, confidential information (P). In the example in Figure 2, the memory unit 17 is located inside the terminal device 10, but it may also be located outside the terminal device 10.

[0042] ● We will explain the information processing equipment (server device) in detail. The information processing device 20 includes a receiving unit 21, a transmitting unit 22, a prompt generation unit 23, a response generation unit 24, a language model 25, and a storage unit 26.

[0043] The receiving unit 21 receives question information (R) from the terminal device 10 via the network 30.

[0044] The prompt generation unit 23 generates a first prompt (prompt1=G+R) using the received question information (R) and general information (G) related to the question information (R).

[0045] The response generation unit 24 inputs the first prompt (prompt1=G+R) into the language model 25 (LLM1) to generate the first response information (A1=LLM1(prompt1)).

[0046] Language model 25 (LLM1) is, for example, a large-scale language model. Examples of large-scale language models include self-attention-based large-scale language models. However, it is not limited to the large-scale language models mentioned above.

[0047] The transmitting unit 22 transmits the first response information (A1) to the terminal device 10 via the network 30.

[0048] The storage unit 26 is a circuit with memory. The storage unit 26 stores information such as general information. In the example in Figure 2, the storage unit 26 is located inside the information processing device (server device) 20, but it may also be located outside the information processing device 20. In that case, for example, a database or a server computer could be considered.

[0049] [Device operation] ●The operation of the terminal device will be explained in detail. Next, the operation of the terminal device in Embodiment 1 will be explained using Figure 3. Figure 3 is a diagram illustrating an example of the operation of the terminal device in Embodiment 1. In the following explanation, the diagram will be referred to as appropriate. In Embodiment 1, the information processing method of the terminal device is implemented by operating the terminal device. Therefore, the explanation of the information processing method in Embodiment 1 will be replaced by the following explanation of the operation of the terminal device.

[0050] As shown in Figure 3, the transmission unit 11 first obtains question information (R) representing a question entered by the user using at least the operation unit 15 (step A1).

[0051] Next, the transmission unit 11 transmits the question information (R) to the information processing device 20 (server device) which has a language model 25 (first language model: LLM1) that answers the question (step A2).

[0052] Next, the receiving unit 12 receives the first response information (A1=LLM1(prompt1)) obtained by inputting the first prompt (prompt1=G+R), which is generated using the question information (R) and general information (G:general) related to the question information (R), into the language model 25 (LLM1) via the network 30 from the information processing device 20 (step A3).

[0053] Next, the prompt generation unit 13 generates a second prompt (prompt2=R+A1+P) using the question information (R), the first answer information (A1), and the secret information (P) associated with the pre-set question information (step A4).

[0054] Next, the response generation unit 14 inputs a second prompt (prompt2) to the language model 16 (LLM2) to generate a second response information (A2=LLM2(prompt2)) (step A5). Subsequently, the second response information (A2) is presented to the user using the user interface of the terminal device 10.

[0055] ●The operation of the information processing device (server device) will be explained in detail. Next, the operation of the information processing device (server device) in Embodiment 1 will be explained using Figure 4. Figure 4 is a diagram illustrating an example of the operation of the information processing device (server device) in Embodiment 1. In the following explanation, the diagram will be referred to as appropriate. In Embodiment 1, the information processing method of the information processing device (server device) is implemented by operating the information processing device (server device). Therefore, the explanation of the information processing method in Embodiment 1 will be replaced by the following explanation of the operation of the information processing device (server device).

[0056] The receiving unit 21 first receives question information (R) from the terminal device 10 via the network 30 (step B1).

[0057] Next, the prompt generation unit 23 generates a first prompt (prompt1=G+R) using the received question information (R) and general information (G) related to the question information (R) (step B2).

[0058] Next, the response generation unit 24 inputs the first prompt (prompt1=G+R) to the language model 25 (LLM1) to generate the first response information (A1=LLM1(prompt1)) (step B3).

[0059] Next, the transmission unit 22 transmits the first response information (A1) to the terminal device 10 via the network 30 (step B4).

[0060] [Effects of Embodiment 1] As described above, according to Embodiment 1, in processing using the language model, confidential information is handled by the terminal device without being transmitted to the cloud side (information processing device (server device)), thus preventing the leakage of confidential information.

[0061] Furthermore, since the cloud first obtains the first answer information using the question information and general information, and the terminal device obtains the second answer information using the question information, the first answer information, and confidential information, the processing load on the language model on the terminal device can be reduced. [Examples]

[0062] In Example 1, we will explain, using Figure 5, a case where a decision is required that takes into account not only the law but also internal company regulations. Figure 5 is a diagram illustrating an example of the operation of the system in Example 1.

[0063] Specifically, general information (G) is defined as a legal document database (DB) pre-stored in the memory unit 26. The legal document DB is, for example, a massive amount of data, exceeding 60,000 characters. Confidential information (P) is defined as an internal regulations database (DB) pre-stored in the memory unit 17. The internal regulations DB is, for example, a small amount of data, such as 1,000 characters.

[0064] In step A1, first, the user enters the question "What should I be careful about when I want to have a side job?" (25 characters) using the operation unit 15 of the terminal device 10.

[0065] Next, in step A2, the transmission unit 11 of the terminal device 10 transmits question information (R) corresponding to the question to the information processing device 20 (server device) via the network 30.

[0066] Next, in step B1, the receiving unit 21 of the information processing device 20 (server device) receives question information (R) from the terminal device 10 via the network 30.

[0067] Next, in step B2, the prompt generation unit 23 of the information processing device 20 (server device) generates a first prompt (prompt1=G+R) using the received question information (R) and general information (G) related to the question information (R).

[0068] For example, the general information (G) might be a text document listing legal documents related to tax returns, item by item, and the question information (R) might be the question entered into terminal device 10, "What should I be careful about when I want to have a side job?" However, the first prompt is not limited to the content described above.

[0069] Next, in step B3, the answer generation unit 24 of the information processing device 20 (server device) inputs the first prompt (prompt1=G+R) to the language model 25 (LLM1) to generate the first answer information (A1=LLM1(prompt1)). The first answer information is, for example, "Tax return etc..." (200 characters).

[0070] Next, in step B4, the transmission unit 22 of the information processing device 20 (server device) transmits the first response information (A1) to the terminal device 10 via the network 30.

[0071] Next, in step A3, the receiving unit 12 of the terminal device 10 receives the first response information (A1=LLM1(prompt1)) from the information processing device 20.

[0072] Next, in step A4, the prompt generation unit 13 of the terminal device 10 generates a second prompt (prompt2=R+A1+P) using the question information (R), the first answer information (A1), and the secret information (P) associated with the pre-set question information.

[0073] For example, the second prompt is generated by setting the question information (R) to "What should I be careful about when I want to have a second job?", using the first answer information (A1) as is, with the text obtained from language model 25 (LLM1) being "If you receive a salary from more than one employer, you will need to file a tax return", and using confidential information (P) as text listing the employment regulations. However, the second prompt is not limited to the content described above.

[0074] Next, in step A5, the response generation unit 14 of the terminal device 10 inputs the second prompt (prompt2) to the language model 16 (LLM2) to generate the second response information (A2=LLM2(prompt2)).

[0075] The first type of information provided in the response is, for example, "In addition to filing a tax return, you will also need to apply for permission to work a second job within the company..."

[0076] In this way, since the confidential information (P) is handled by the terminal device 10 without being transmitted to the cloud side (information processing device (server device)), the leakage of confidential information can be prevented.

[0077] Furthermore, conventionally, all processing is performed on the information processing device (server device) on the cloud side, so the conventional prompt is generated using general information (G), question information (R), and secret information (P) (prompt = G + R + P). Therefore, the number of characters in the conventional prompt is 61,025. Consequently, if the terminal device processes the conventional prompt, the processing load on the language model increases.

[0078] However, by using the method described in Embodiment 1, the processing load of the language model on the terminal device 10 can be reduced. Specifically, the information processing device 20 (server device) on the cloud generates a first prompt (prompt1=G+R: 60025 characters), and uses the first prompt (prompt1) to generate the first response information (A1). Note that there is no problem even if the processing load of the language model on the information processing device 20 (server device) on the cloud is large. Also, unlike conventional methods, the first prompt (prompt1) does not contain secret information (P).

[0079] Furthermore, the terminal device 10 obtains the second answer information (A2) using the first answer information (A1), the question information (R), and the secret information (P), thus reducing the processing load on the language model on the terminal device 10. Specifically, the number of characters in the second prompt (prompt2=A1+R+P) generated by the terminal device 10 is, for example, around 1225 characters, thus reducing the processing load on the language model on the terminal device 10.

[0080] [program] ●This section explains the program for the terminal device. The program for the terminal device in Embodiment 1 is any program that causes the computer to execute steps A1 to A5 shown in Figure 3. By installing and running this program on the computer, the terminal device and information processing method in Embodiment 1 can be realized. In this case, the computer's processor functions as a prompt generation unit 13, an answer generation unit 14, an operation unit 15, and a language model 16, and performs the processing.

[0081] Furthermore, the program for the terminal device in Embodiment 1 may be executed by a computer system constructed by multiple computers. In this case, for example, each computer may function as one of the following: a prompt generation unit 13, an answer generation unit 14, an operation unit 15, or a language model 16.

[0082] ●This section explains the program for the information processing device (server device). The program for the information processing device (server device) in Embodiment 1 is any program that causes a computer to execute steps B1 to B4 shown in Figure 4. By installing and executing this program on a computer, the information processing device (server device) and information processing method in Embodiment 1 can be realized. In this case, the computer's processor functions as a prompt generation unit 23, an answer generation unit 24, and a language model 25, and performs the processing.

[0083] Furthermore, the program for the terminal device in Embodiment 1 may be executed by a computer system constructed by multiple computers. In this case, for example, each computer may function as either the prompt generation unit 23, the answer generation unit 24, or the language model 25.

[0084] (Embodiment 2) The configuration of the terminal device in Embodiment 2 will be explained using Figure 6. Figure 6 is a diagram illustrating an example of the terminal device in Embodiment 2.

[0085] [Device configuration] The terminal device shown in Figure 6 is a device that, in processing using a language model, handles confidential information solely on the terminal device side and reduces the processing load on the terminal device. Also, as shown in Figure 6, the terminal device 60 has a transmission unit 61, a reception unit 62, a writing unit 63, a prompt generation unit 64, and a response generation unit 65.

[0086] The transmitting unit 61 transmits at least question information representing a question entered by a user using an input device to an information processing device (server device) having a first language model that outputs an answer to the question.

[0087] The receiving unit 62 reads cache information from the memory used by the first language model, which was generated when the first prompt, generated using the question information and general information related to the question information, was input to the first language model to generate the first answer information, and receives the cache information and general information from the information processing device. Note that the generation of the first answer information may be terminated midway. The cache information may be constructed up until just before the answer information is output.

[0088] The write unit 63 writes the cache information to the memory used by the second language model.

[0089] The prompt generation unit 64 generates a second prompt using the question information, general information, and secret information related to the pre-configured question information.

[0090] The response generation unit 65 inputs the second prompt to the second language model to generate the second response information.

[0091] Thus, in Embodiment 2, in processing using the language model, confidential information is handled by the terminal device without being transmitted to the cloud side (information processing device (server device)), thus preventing the leakage of confidential information.

[0092] Furthermore, on the cloud side, the first answer information is calculated using the question information and general information, and the information (cache information) used in the memory (KV cache) of the first language model, which was generated when the first answer information was created, is written to the memory (KV cache) used in the second language model of the terminal device. This reduces the amount of processing required to generate cache information again on the terminal device side.

[0093] [System Configuration] Next, the configuration of the terminal device 60 in Embodiment 2 will be described in more detail using Figure 7. Figure 7 is a diagram illustrating an example of a system having the terminal device of Embodiment 2.

[0094] As shown in Figure 7, the system 200 in Embodiment 2 includes a terminal device 60, an information processing device 70 (server device), and a network 30.

[0095] The terminal device 60 is, for example, a programmable device such as a CPU (Central Processing Unit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or a circuit equipped with one or more of these, an information processing device located on the edge side of a server computer, personal computer, mobile terminal, etc.

[0096] The information processing device 70 (server device) is, for example, a CPU, a programmable device such as an FPGA, a GPU, or a circuit equipped with one or more of these, or an information processing device located on the cloud side such as a server computer.

[0097] Network 30 is a general network constructed using communication lines such as the Internet, LAN (Local Area Network), dedicated lines, telephone lines, corporate networks, mobile communication networks, Bluetooth (registered trademark), and Wi-Fi (Wireless Fidelity) (registered trademark).

[0098] ●The terminal device will be explained in detail. The terminal device 60 includes an operation unit 66, a transmission unit 61, a reception unit 62, a writing unit 63, a prompt generation unit 64, an answer generation unit 65, a language model 67, and a storage unit 68.

[0099] The operation unit 66 is a user interface for inputting questions using input devices such as a touch panel, mouse, or keyboard, which are provided on the terminal device 60. However, the operation unit 66 may be provided outside the terminal device 60.

[0100] First, the transmitting unit 61 obtains question information (R: request) representing a question entered by the user using at least the operating unit 66. Next, the transmitting unit 61 transmits this information to an information processing device 70 (server device) which has a language model 76 (first language model: LLM1) that answers the question, as will be described later.

[0101] The receiving unit 62 reads cache information (C: cache) from the memory used by the first language model 76, which was generated when it inputs the first prompt (prompt1=G+R), generated using the question information (R) and general information (G: general) related to the question information (R), into the language model 76 (LLM1) to generate the first answer information (A1=LLM1(prompt1)), and receives the cache information (C) and general information (G) from the information processing device 70.

[0102] The write unit 63 writes the cache information (C) to the memory used by the language model 67 (second language model: LLM2).

[0103] The first language model (LLM1) and the second language model (LLM2) are the same language model. Furthermore, the cache information (C) is the information stored in the KV cache used by the first language model (LLM1).

[0104] The prompt generation unit 64 generates a second prompt (prompt2=G+R+P) using the question information (R), general information (G), and secret information (P: private) related to the pre-configured question information. Since the beginning of both the first prompt (Prompt1) and the second prompt (prompt2) is G+R and they match, processing can be reduced by sharing cache information (C) in the same language model.

[0105] The response generation unit 65 inputs the second prompt (prompt2) to the language model 67 (second language model: LLM2) to generate the second response information (A2=LLM2(prompt2)).

[0106] Language model 67 (LLM2) is, for example, a large-scale language model. Examples of large-scale language models include self-attention-based large-scale language models. However, it is not limited to the large-scale language models mentioned above.

[0107] The storage unit 68 is a circuit with memory. The storage unit 68 stores information such as, for example, confidential information (P). In the example in Figure 7, the storage unit 68 is located inside the terminal device 60, but it may also be located outside the terminal device 60.

[0108] ● We will explain the information processing equipment (server device) in detail. The information processing device 70 includes a receiving unit 71, a transmitting unit 72, a prompt generation unit 73, a response generation unit 74, a reading unit 75, a language model 76, and a storage unit 77.

[0109] The receiving unit 71 receives question information (R) from the terminal device 60 via the network 30.

[0110] The prompt generation unit 73 generates a first prompt (prompt1=G+R) using the received question information (R) and general information (G) related to the question information (R).

[0111] The response generation unit 74 inputs the first prompt (prompt1=G+R) to the language model 76 (LLM1) to generate the first response information (A1=LLM1(prompt1)).

[0112] Language model 76 (LLM1) is, for example, a large-scale language model. Examples of large-scale language models include self-attention-based large-scale language models. However, it is not limited to the large-scale language models mentioned above.

[0113] The reading unit 75 reads cache information (C) from the memory used by the first language model 76 (LLM1), which was generated when the first answer information (A1) was generated.

[0114] The transmitting unit 72 transmits cache information (C) and general information (G) to the terminal device 60 via the network 30.

[0115] The storage unit 77 is a circuit with memory. The storage unit 77 stores information such as general information. In the example in Figure 7, the storage unit 77 is located inside the information processing device (server device) 70, but it may also be located outside the information processing device 70. In that case, for example, a database or a server computer could be considered.

[0116] [Device operation] ●The operation of the terminal device will be explained in detail. Next, the operation of the terminal device in Embodiment 2 will be described using Figure 8. Figure 8 is a diagram illustrating an example of the operation of the terminal device in Embodiment 2. In the following description, the diagram will be referred to as appropriate. In Embodiment 2, the information processing method of the terminal device is implemented by operating the terminal device. Therefore, the explanation of the information processing method in Embodiment 2 will be replaced by the following explanation of the operation of the terminal device.

[0117] As shown in Figure 8, the transmitting unit 61 first obtains question information (R) representing a question entered by the user using at least the operating unit 66 (step C1).

[0118] Next, the transmission unit 61 transmits the question information (R) to the information processing device 70 (server device) which has a language model 76 (first language model: LLM1) that answers the question (step C2).

[0119] Next, the receiving unit 62 receives cache information (C) and general information (G) from the information processing device 70, which are obtained by reading cache information (C) from the memory used by the language model 76 when the first prompt (prompt1=G+R), generated using the question information (R) and general information (G) related to the question information (R), is input to the language model 76 (LLM1) to generate the first answer information (A1=LLM1(prompt1)). (Step C3)

[0120] Next, the write unit 63 writes the cache information (C) to the memory used by the language model 67 (LLM2) (step C4).

[0121] Next, the prompt generation unit 64 generates a second prompt (prompt2=G+R+P) using the question information (R), general information (G), and secret information (P) related to the pre-set question information (step C5).

[0122] Next, the response generation unit 65 inputs a second prompt (prompt2) to the language model 67 (LLM2) to generate second response information (A2=LLM2(prompt2)) (step C6). Subsequently, the second response information (A2) is presented to the user using the user interface of the terminal device 60.

[0123] ●The operation of the information processing device (server device) will be explained in detail. Next, the operation of the information processing device (server device) in Embodiment 2 will be explained using Figure 9. Figure 9 is a diagram illustrating an example of the operation of the information processing device (server device) in Embodiment 2. In the following explanation, the diagram will be referred to as appropriate. In Embodiment 2, the information processing method of the information processing device (server device) is implemented by operating the information processing device (server device). Therefore, the explanation of the information processing method in Embodiment 2 will be replaced by the following explanation of the operation of the information processing device (server device).

[0124] The receiving unit 71 first receives question information (R) from the terminal device 60 via the network 30 (step D1).

[0125] Next, the prompt generation unit 73 generates a first prompt (prompt1=G+R) using the received question information (R) and general information (G) related to the question information (R) (step D2).

[0126] Next, the response generation unit 74 inputs the first prompt (prompt1=G+R) to the language model 76 (LLM1) to generate the first response information (A1=LLM1(prompt1)) (step D3).

[0127] Next, the reading unit 75 reads cache information (C) from the memory used by the first language model 76 (LLM1), which was generated when the first answer information (A1) was generated (step D4).

[0128] Next, the transmission unit 72 transmits the cache information (C) and general information (G) to the terminal device 60 via the network 30 (step D5).

[0129] [Effects of Embodiment 2] As described above, according to Embodiment 2, in processing using the language model, confidential information is handled by the terminal device without being transmitted to the cloud side (information processing device (server device)), thus preventing the leakage of confidential information.

[0130] Furthermore, on the cloud side, the first answer information is calculated using the question information and general information, and the information (cache information) used in the memory (KV cache) of the first language model, which was generated when the first answer information was produced, is written to the memory (KV cache) used in the second language model of the terminal device. This reduces the amount of processing required to generate cache information again on the terminal device side. [Examples]

[0131] In Example 2, we will explain, using Figure 10, a case where a decision is required that takes into account not only the law but also internal company regulations. Figure 10 is a diagram illustrating an example of the operation of the system in Example 2.

[0132] Specifically, general information (G) is defined as a legal document database (DB) pre-stored in the memory unit 26. The legal document DB contains a large amount of data, for example, more than 60,000 characters. Confidential information (P) is defined as an internal regulations database (DB) pre-stored in the memory unit 17. The internal regulations DB contains data, for example, 1,000 characters.

[0133] In step C1, first, the user enters the question "What should I be careful about when I want to have a side job?" (25 characters) using the operation unit 66 of the terminal device 60.

[0134] Next, in step C2, the transmission unit 61 of the terminal device 60 transmits question information (R) corresponding to the question to the information processing device 70 (server device) via the network 30.

[0135] Next, in step D1, the receiving unit 71 of the information processing device 70 (server device) receives question information (R) from the terminal device 60 via the network 30.

[0136] Next, in step D2, the prompt generation unit 73 of the information processing device 70 (server device) generates a first prompt (prompt1=G+R) using the received question information (R) and general information (G) related to the question information (R).

[0137] Next, in step D3, the answer generation unit 74 of the information processing device 70 (server device) inputs the first prompt (prompt1=G+R) to the language model 76 (LLM1) to generate the first answer information (A1=LLM1(prompt1)). The first answer information is, for example, "Tax return etc..." (200 characters). Note that the generation of the first answer information may be terminated midway.

[0138] Next, in step D4, the reading unit 75 of the information processing device 70 (server device) reads cache information (C) from the memory used by the first language model 76 (LLM1) that was generated when the first answer information (A1) was generated.

[0139] Next, in step D5, the transmission unit 72 of the information processing device 70 (server device) transmits the cache information (C) and general information (G) to the terminal device 60 via the network 30.

[0140] Next, in step C3, the receiving unit 62 of the terminal device 60 receives cache information (C) and general information (G) from the information processing device 70.

[0141] Next, in step C4, the writing unit 63 of the terminal device 60 writes the cache information (C) to the memory used by the language model 67 (LLM2).

[0142] Next, in step C5, the prompt generation unit 64 of the terminal device 60 generates a second prompt (prompt2=G+R+P) using the question information (R), general information (G), and secret information (P) related to the pre-set question information.

[0143] For example, the second prompt is generated using the following format: General Information (G) is a text listing legal documents related to tax returns, Question Information (R) is the question entered into terminal device 10, "What should I be careful about when I want to have a side job?", and Confidential Information (P) is a text listing documents related to employment regulations. However, the second prompt is not limited to the content described above.

[0144] Next, in step C6, the response generation unit 65 of the terminal device 60 inputs the second prompt (prompt2) to the language model 16 (LLM2) to generate the second response information (A2=LLM2(prompt2)).

[0145] [program] ●This section explains the program for the terminal device. The program for the terminal device in Embodiment 2 is any program that causes the computer to execute steps C1 to C6 shown in Figure 8. By installing and running this program on the computer, the terminal device and information processing method in Embodiment 2 can be realized. In this case, the computer's processor functions as a writing unit 63, a prompt generation unit 64, a response generation unit 65, an operation unit 66, and a language model 67, and performs the processing.

[0146] Furthermore, the program for the terminal device in Embodiment 2 may be executed by a computer system constructed by multiple computers. In this case, for example, each computer may function as one of the following: the writing unit 63, the prompt generation unit 64, the answer generation unit 65, the operation unit 66, or the language model 67.

[0147] ●This section explains the program for the information processing device (server device). The program for the information processing device (server device) in Embodiment 2 is any program that causes the computer to execute steps B1 to B4 shown in Figure 9. By installing and executing this program on the computer, the information processing device (server device) and information processing method in Embodiment 1 can be realized. In this case, the computer's processor functions as a prompt generation unit 73, an answer generation unit 74, a reading unit 75, and a language model 76, and performs the processing.

[0148] Furthermore, the program for the terminal device in Embodiment 2 may be executed by a computer system constructed by multiple computers. In this case, for example, each computer may function as one of the following: a prompt generation unit 73, an answer generation unit 74, a reading unit 75, or a language model 76.

[0149] [Physical configuration] Here, a computer that implements a terminal device and an information processing device (server device) by executing the programs in Embodiments 1 and 2 will be described using Figure 11. Figure 11 is a diagram illustrating an example of a computer that implements a terminal device and an information processing device (server device) in Embodiments 1 and 2.

[0150] As shown in Figure 11, the computer 110 comprises a CPU (Central Processing Unit) 111, main memory 112, storage device 113, input interface 114, display controller 115, data reader / writer 116, and communication interface 117. These components are connected to each other via a bus 121, enabling data communication. In addition to the CPU 111, or in place of the CPU 111, the computer 110 may also include a GPU or FPGA.

[0151] The CPU 111 loads the program in the embodiment, which consists of a set of codes stored in the storage device 113, into the main memory 112, and performs various calculations by executing each code in a predetermined order. The main memory 112 is typically a volatile storage device such as DRAM (Dynamic Random Access Memory).

[0152] Furthermore, the program in this embodiment is provided stored on a computer-readable recording medium 120. The program in this embodiment may also be distributed over the Internet via a communication interface 117.

[0153] Specific examples of the storage device 113 include hard disk drives and semiconductor storage devices such as flash memory. The input interface 114 mediates data transmission between the CPU 111 and input devices 118 such as a keyboard and mouse. The display controller 115 is connected to the display device 119 and controls the display on the display device 119.

[0154] The data reader / writer 116 mediates data transmission between the CPU 111 and the recording medium 120, reads programs from the recording medium 120, and writes processing results from the computer 110 to the recording medium 120. The communication interface 117 mediates data transmission between the CPU 111 and other computers.

[0155] Specific examples of the recording medium 120 include general-purpose semiconductor memory devices such as CF (Compact Flash®) and SD (Secure Digital), magnetic recording media such as Flexible Disks, and optical recording media such as CD-ROMs (Compact Disk Read Only Memory).

[0156] Furthermore, the terminal device and information processing device (server device) in Embodiments 1 and 2 can be implemented not by a computer with a program installed, but by using hardware corresponding to each part, such as electronic circuits. Moreover, the terminal device and information processing device (server device) may be partially implemented by a program and the remaining part by hardware. In the embodiments, the computer is not limited to the computer shown in Figure 11.

[0157] Although the invention has been described above with reference to embodiments, the invention is not limited to the embodiments described above. Various modifications to the structure and details of the invention can be made that will be understood by those skilled in the art within the scope of the invention. [Industrial applicability]

[0158] According to the above description, processing using a language model allows confidential information to be handled solely by the terminal device, while also reducing the processing load on the terminal device. Furthermore, it is useful in fields where a language processing model is required. [Explanation of symbols]

[0159] 10, 60 Terminal devices 11, 61 Transmitter 12, 62 Receiver 13, 64 Prompt generation unit 14, 65 Answer generation part 15, 66 Operation section 16, 67 Language Models 17, 68 Memory section 20, 70 Information Processing Equipment 21, 71 Receiving section 22, 72 Transmitter 23, 73 Prompt display section 24, 74 Answer generation part 25, 76 Language Models 26, 77 Memory section 30 Networks 63 Writing section 75 Reading section 100, 200 systems 110 Computer 111 CPU 112 Main Memory 113 Storage device 114 Input Interface 115 Display Controller 116 Data Readers / Writers 117 Communication Interface 118 Input devices 119 Display device 120 recording media 121 Bus

Claims

1. At a minimum, a transmission means for transmitting question information representing a question entered by a user using an input device to an information processing device having a first language model that answers the question, A receiving means that receives first response information obtained by inputting a first prompt, generated using the aforementioned question information and general information related to the aforementioned question information, into the first language model, from the information processing device. A prompt generation means that generates a second prompt using the aforementioned question information, the first answer information, and pre-set secret information related to the aforementioned question information, A response generation means that inputs the second prompt to a second language model to generate second response information, A terminal device having the following features.

2. Computers At a minimum, question information representing a question entered by a user using an input device is transmitted to an information processing device having a first language model that answers the question. The first response information obtained by inputting a first prompt, generated using the aforementioned question information and general information related to the aforementioned question information, into the first language model is received from the information processing device. A second prompt is generated using the aforementioned question information, the first answer information, and the pre-configured secret information related to the aforementioned question information. The second prompt is input to the second language model to generate the second response information. Information processing methods.

3. On the computer, At a minimum, the system transmits question information representing a question entered by a user using an input device to an information processing device having a first language model that answers the question. The first response information obtained by inputting a first prompt, generated using the aforementioned question information and general information related to the aforementioned question information, into the first language model is received from the information processing device. Using the aforementioned question information, the first answer information, and the pre-configured secret information related to the aforementioned question information, a second prompt is generated. The second prompt is input to the second language model to generate the second response information. program.

4. The terminal device transmits question information, which represents a question entered by the user using an input device, to a server device having a first language model that answers the question. The aforementioned server device The terminal device receives the question information, A first prompt is generated using the aforementioned question information and general information related to the aforementioned question information. The first prompt is input to the first language model to generate the first response information. The first response information is transmitted to the terminal device. The aforementioned terminal device The server device receives the first response information, A second prompt is generated using the aforementioned question information, the first answer information, and the pre-configured secret information related to the aforementioned question information. The second prompt is input to the second language model to generate the second response information. system.

5. At a minimum, a transmission means for transmitting question information representing a question entered by a user using an input device to an information processing device having a first language model that answers the question, A receiving means for receiving the cache information and the general information from the information processing device, which are generated when a first prompt, generated using the aforementioned question information and general information related to the aforementioned question information, is input to the first language model to generate first answer information, and the cache information is read from the memory used by the first language model. A writing means for writing the aforementioned cache information to memory used by the second language model, A prompt generation means that generates a second prompt using the aforementioned question information, the aforementioned general information, and pre-set secret information related to the aforementioned question information, A response generation means that inputs the second prompt to the second language model to generate second response information, A terminal device having the following features.

6. The first language model and the second language model are the same language model, and the cache information is the information in the KV cache. The terminal device according to claim 5.

7. Computers At a minimum, question information representing a question entered by a user using an input device is transmitted to an information processing device having a first language model that answers the question. When a first prompt generated using the aforementioned question information and general information related to the aforementioned question information is input to the first language model to generate first answer information, the cache information generated when the cache information is read from the memory used by the first language model, and the cache information and the general information are received from the information processing device. A writing means for writing the aforementioned cache information to memory used by the second language model, A second prompt is generated using the aforementioned question information, the aforementioned general information, and the pre-configured secret information related to the aforementioned question information. The second prompt is input to the second language model to generate the second response information. Information processing methods.

8. On the computer, At a minimum, the system transmits question information representing a question entered by a user using an input device to an information processing device having a first language model that answers the question. When a first prompt generated using the aforementioned question information and general information related to the aforementioned question information is input to the first language model to generate first answer information, the cache information generated when the cache information is read from the memory used by the first language model, and the cache information and the general information are received from the information processing device. A writing means for writing the aforementioned cache information to memory used by the second language model, Using the aforementioned question information, the aforementioned general information, and the pre-configured secret information related to the aforementioned question information, a second prompt is generated. The second prompt is input to the second language model to generate the second response information. program.

9. The terminal device transmits question information, which represents a question entered by the user using an input device, to a server device having a first language model that answers the question. The aforementioned server device The terminal device receives the question information, A first prompt is generated using the aforementioned question information and general information related to the aforementioned question information. The first prompt is input to the first language model to generate the first response information. When generating the first response information, the cache information used by the first language model is read from the memory, The cache information and the general information are transmitted to the terminal device. The terminal device, The server device receives the cache information and the general information, The write operation involves writing the aforementioned cache information to the memory used by the second language model. A second prompt is generated using the aforementioned question information, the aforementioned general information, and the pre-configured secret information related to the aforementioned question information. The second prompt is input to the second language model to generate the second response information. system.

Images