Language model utilization device, language model utilization method, and language model utilization program

Abstract

The aim is to reduce the amount of information in prompts, thereby lowering the utilization rate of large-scale language models and reducing the costs associated with using them. [Solution] The language model utilization device 100 utilizes a first language model 61 and a second language model 62 which is smaller in scale than the first language model. The gatekeeper unit 110 inputs the first prompt 51 to the second language model 62 and acquires the second output information output from the second language model 62. The gatekeeper unit 110 acquires the second output information as the second prompt 52 obtained by information compression of the first prompt 51. The answer generation unit 120 inputs the second prompt 52 to the first language model 61 and acquires the first output information output from the first language model 61 as the answer 53 to the first prompt 51.

Description

Technical Field

[0001] The present disclosure relates to a language model utilization device, a language model utilization method, and a language model utilization program.

Background Art

[0002] In recent years, systems for performing natural language processing such as question-and-answer by utilizing large language models called LLM have been provided. LLM is an abbreviation for Large Language Models. An LLM is a natural language processing model learned using a large amount of text data. When an LLM is applied to a system for performing question-and-answer, when a prompt including a question sentence is input to the LLM, an answer sentence for the question sentence is output from the LLM.

[0003] On the other hand, as the amount of information in the prompt input to the LLM increases, an increase in the cost of using the LLM has become a problem. In particular, in the case of an LLM on the cloud, there are cases where the usage fee becomes extremely high depending on the amount of information in the prompt.

[0004] Patent Document 1 discloses a technique for efficiently learning an SLM to be added to an LLM so that the output sentence from the SLM reflects knowledge of a specific area. SLM is an abbreviation for Small Language Models.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] While the technology described in Patent Document 1 can reduce the computational load during training, it cannot reduce the computational load during answer generation. Therefore, as the amount of prompt information input to the LLM increases, there is a problem in that the cost of using the LLM increases.

[0007] This disclosure aims to reduce the cost of using the LLM by decreasing the amount of prompt information entered into the LLM and thereby lowering the LLM's utilization rate. [Means for solving the problem]

[0008] The language model utilization device related to this disclosure is In a language model utilization device that utilizes a first language model and a second language model smaller in scale than the first language model, A gatekeeper unit that acquires a first prompt, inputs the first prompt to the second language model, and acquires the second output information, which is information output from the second language model, as a second prompt obtained by information compression of the first prompt. A response generation unit inputs the second prompt to the first language model and obtains the first output information output from the first language model as a response to the first prompt. It is equipped with. [Effects of the Invention]

[0009] The language model utilization device described in this disclosure utilizes a first language model and a second language model that is smaller in scale than the first language model. The language model utilization device described in this disclosure uses the second language model to reduce the amount of prompt information input to the first language model. Therefore, the language model utilization device described in this disclosure has the effect of reducing the utilization rate of the first language model and reducing the costs associated with using the first language model. [Brief explanation of the drawing]

[0010] [Figure 1] A diagram showing an example configuration of a language model utilization device according to Embodiment 1. [Figure 2] A diagram showing an example of the functional configuration of a language model utilization system according to Embodiment 1. [Figure 3] A flowchart illustrating an example of language model utilization processing by the language model utilization device according to Embodiment 1. [Figure 4] A schematic diagram showing a specific example of information compression using a language model utilization device according to Embodiment 1. [Figure 5] A schematic diagram showing a specific example 2 of information compression using a language model utilization device according to Embodiment 1. [Figure 6] This figure shows an example of the configuration of a language model utilization device according to a modified embodiment of Embodiment 1. [Figure 7] A flowchart illustrating an example of language model utilization processing by the language model utilization device according to Embodiment 2. [Modes for carrying out the invention]

[0011] The following description of this embodiment will be illustrated with reference to the figures. In each figure, identical or corresponding parts are denoted by the same reference numerals. In the description of the embodiment, the explanation of identical or corresponding parts will be omitted or simplified as appropriate. The arrows in the figures mainly indicate the flow of data or processing.

[0012] Embodiment 1. ***Explanation of the structure*** Figure 1 is a diagram showing an example configuration of the language model utilization device 100 according to this embodiment. Figure 2 shows an example of the functional configuration of the language model utilization system 500 according to this embodiment. The language model utilization device 100 is a computer. The language model utilization device 100 utilizes a first language model 61 and a second language model 62 which is smaller in scale than the first language model 61. The language model utilization device 100 is, for example, a question and answer system that uses the first language model 61 and the second language model 62.

[0013] The language model utilization system 500 comprises a language model utilization device 100 and a first language model 61 that communicates with the language model utilization device 100. In this embodiment, the second language model 62 is provided in the language model utilization device 100. Note that the second language model 62 may be provided outside the language model utilization device 100 so as to be communicable with the language model utilization device 100.

[0014] The first language model 61 is also called an artificial intelligence 200. The first language model 61 is, for example, a type of generative AI specialized in natural language processing. AI is an abbreviation of Artificial Intelligence. The first language model 61 is a large-scale LLM that realizes more advanced language understanding by learning from a vast amount of text data. Specific examples of the first language model 61 include GPT-4, PaLM2, or LLaMa2. GPT is an abbreviation of Generative Pre-trained Transformer. PaLM is an abbreviation of Pathways Language Model. LLaMa is an abbreviation of Large Language Model meta AI. The first language model 61 is, for example, provided on the cloud. As the amount of information in the input prompt increases, the usage fee of the first language model 61 also becomes high. The first language model 61 is also referred to as the first LLM.

[0015] The second language model 62 is also called an artificial intelligence 200 and is a type of generative AI specialized in natural language processing. In this embodiment, a language model smaller in scale than the first language model 61 is assumed as the second language model 62. Specific examples of the second language model 62 include Claude, Gemini, or CodeLLama. The second language model 62 is, for example, provided in an on-premises environment. In the present embodiment, the second language model 62 is provided inside the language model utilization device 100, but it may be provided outside the language model utilization device 100. When the second language model 62 is provided outside the language model utilization device 100, a usage fee may be charged, but it can be used at a lower cost than the first language model 61. Also, when the second language model 62 is provided in an on-premises environment, it can be used at a low cost with only the cost of traffic. The second language model 62 is also referred to as the second LLM or the local LLM.

[0016] The language model utilization device 100 includes a processor 910 and also includes other hardware such as a memory 921, an auxiliary storage device 922, an input interface 930, an output interface 940, and a communication device 950. The processor 910 is connected to the other hardware via signal lines and controls these other hardware.

[0017] As functional elements, the language model utilization device 100 includes a gatekeeper unit 110, an answer generation unit 120, and a storage unit 150. The storage unit 150 stores the second language model 62, the first prompt 51, the second prompt 52, and the answer 53.

[0018] The functions of the gatekeeper unit 110 and the answer generation unit 120 are realized by software. The storage unit 150 is provided in the memory 921. Note that the storage unit 150 may be provided in the auxiliary storage device 922 or may be provided in a distributed manner in the memory 921 and the auxiliary storage device 922.

[0019] The processor 910 is a device that executes a language model utilization program. The language model utilization program is a program that realizes the functions of the gatekeeper unit 110 and the answer generation unit 120. The processor 910 is an integrated circuit (IC) that performs arithmetic processing. Specific examples of the processor 910 include CPUs, DSPs, and GPUs. IC stands for Integrated Circuit. CPU stands for Central Processing Unit. DSP stands for Digital Signal Processor. GPU stands for Graphics Processing Unit.

[0020] Memory 921 is a storage device that temporarily stores data. Specific examples of memory 921 are SRAM or DRAM. SRAM is an abbreviation for Static Random Access Memory. DRAM is an abbreviation for Dynamic Random Access Memory. The auxiliary storage device 922 is a storage device for storing data. A specific example of the auxiliary storage device 922 is an HDD. Alternatively, the auxiliary storage device 922 may be a portable storage medium such as an SD® memory card, CF, NAND flash, flexible disk, optical disk, compact disk, Blu-ray® disc, or DVD. HDD is an abbreviation for Hard Disk Drive. SD® is an abbreviation for Secure Digital. CF is an abbreviation for CompactFlash®. DVD is an abbreviation for Digital Versatile Disk.

[0021] Input interface 930 is a port to which input devices such as a mouse, keyboard, or touch panel are connected. Specifically, input interface 930 is a USB terminal. Alternatively, input interface 930 may be a port connected to a LAN. USB stands for Universal Serial Bus, and LAN stands for Local Area Network.

[0022] Output interface 940 is a port to which the cable of an output device, such as a display, is connected. Specifically, output interface 940 is a USB terminal or an HDMI® terminal. Specifically, the display is an LCD. Output interface 940 is also called the display interface. HDMI® is an abbreviation for High Definition Multimedia Interface. LCD is an abbreviation for Liquid Crystal Display.

[0023] The communication device 950 has a receiver and a transmitter. The communication device 950 is connected to a communication network such as a LAN, the Internet, a telephone line, or Wi-Fi (registered trademark). Specifically, the communication device 950 is a communication chip or NIC. NIC is an abbreviation for Network Interface Card.

[0024] The language model utilization program is executed in the language model utilization device 100. The language model utilization program is loaded into the processor 910 and executed by the processor 910. Memory 921 stores not only the language model utilization program but also the OS. OS is an abbreviation for Operating System. The processor 910 executes the language model utilization program while executing the OS. The language model utilization program and the OS may also be stored in auxiliary storage device 922. The language model utilization program and OS stored in auxiliary storage device 922 are loaded into memory 921 and executed by the processor 910. Note that part or all of the language model utilization program may be incorporated into the OS.

[0025] The language model utilization device 100 may have multiple processors that replace the processor 910. These multiple processors share the task of executing the language model utilization program. Each processor is a device that executes the language model utilization program in the same way as the processor 910.

[0026] The data, information, signal values, and variable values ​​used, processed, or output by the language model utilization program are stored in memory 921, auxiliary storage device 922, or registers or cache memory within the processor 910.

[0027] The word "part" in "gatekeeper unit 110" and "answer generation unit 120" may be replaced with "circuit," "process," "procedure," "process," or "circuitry." The language model utilization program causes the computer to execute the gatekeeper process and the answer generation process. The word "process" in "gatekeeper process" and "answer generation process" may be replaced with "program," "program product," "computer-readable storage medium storing the program," or "computer-readable recording medium recording the program." Furthermore, the language model utilization method is performed by the language model utilization device 100 executing the language model utilization program. The language model utilization program may be provided on a computer-readable recording medium. Alternatively, the language model utilization program may be provided as a program product.

[0028] As shown in Figure 2, the first language model 61 and the second language model 62 are each types of artificial intelligence 200. The artificial intelligence 200 includes an inference unit that performs inference using a trained model. The inference unit takes a prompt as input and outputs a response corresponding to the prompt as output information based on the trained model. In this embodiment, first output information is output from the first language model 61. Second output information is output from the second language model 62.

[0029] Artificial intelligence 200 is, for example, a generative AI. Artificial intelligence 200 is constructed using algorithms such as Transformer, BERT, and GPT. Alternatively, artificial intelligence 200 may be constructed by combining multiple algorithms, including these. BERT is an abbreviation for Bidirectional EncoderRepresentations from Transformers.

[0030] As mentioned above, in this embodiment, the artificial intelligence 200 uses a first language model 61 and a second language model 62 which is smaller in scale than the first language model. Furthermore, although Figure 2 shows a configuration in which the second language model 62 also includes an inference unit, the functions of the inference unit of the second language model 62 may also be combined into the gatekeeper unit 110. RAG is a suitable example of a generative AI design pattern for the second language model 62. RAG stands for Retrieval Augmented Generation. RAG is a technique that searches for and extracts desired information, and then has the generative AI generate an answer based on that information. This allows the generative AI to generate answers not only from previously learned information, but also from unlearned internal company information. Furthermore, the second language model may be an SLM. Alternatively, the second language model may consist of multiple types of language models, including an SLM.

[0031] ***Explanation of operation*** Next, the operation of the language model utilization device 100 according to this embodiment will be described. The operation procedure of the language model utilization device 100 corresponds to the language model utilization method. Furthermore, the program that realizes the language model utilization process, which is the operation of the language model utilization device 100, corresponds to the language model utilization program.

[0032] Figure 3 is a flowchart showing an example of language model utilization processing by the language model utilization device 100 according to this embodiment. The language model utilization process includes gatekeeper processing and response generation processing. In the gatekeeper processing, the gatekeeper unit 110 acquires the first prompt 51 and inputs the first prompt 51 to the second language model 62. The gatekeeper unit 110 acquires the second output information 512, which is information output from the second language model 62, as the second prompt 52 obtained by information compression of the first prompt 51. In the response generation process, the response generation unit 120 inputs the second prompt 52 to the first language model 61 and obtains the first output information 521 output from the first language model 61 as the response 53 to the first prompt 51.

[0033] In this embodiment, as an example of the first prompt 51, a prompt consisting of a question 71 and reference information 72 used to answer the question 71 will be used for explanation. The reference information 72 is, for example, a large amount of source documents used to answer the question 71. Furthermore, the following explanation describes the method of compressing information for Reference Information 72, i.e., a large amount of original documents. However, it is also acceptable to compress the Question 71 and Reference Information 72 together. The method of compressing the Question 71 and Reference Information 72 together will be described later.

[0034] In the gatekeeper processing, the gatekeeper unit 110 obtains a first prompt 51 which includes a question 71. The first prompt 51 is accompanied by reference information 72 which is used to answer the question 71. The gatekeeper unit 110 inputs the first prompt 51 to the second language model 62 and obtains second output information 512 which is information output from the second language model 62. The gatekeeper unit 110 obtains the second output information 512 as a second prompt 52 which includes the question 71 and is accompanied by compressed information 73 obtained by compressing the reference information 72. In the response generation process, the response generation unit 120 inputs the second prompt 52 to the first language model 61 via the communication device 950, and obtains the first output information 521 output from the first language model 61 as the response 53 to the first prompt 51. Specifically, it is as follows:

[0035] <Gatekeeper processing> In step S101, the gatekeeper unit 110 obtains a first prompt 51 which includes a question 71. The first prompt 51 is accompanied by reference information 72 which is used to answer the question 71. Specifically, the gatekeeper unit 110 obtains a first prompt 51 from the user via the input interface 930 or the communication device 950.

[0036] In step S102, the gatekeeper unit 110 inputs the first prompt 51 to the second language model 62 and obtains the second output information 512 output from the second language model 62. The second output information 512 is a second prompt 52 that includes the question sentence 71 and has compressed information 73 added, which is obtained by compressing the reference information 72. The gatekeeper unit 110 obtains the second output information 512 as a second prompt 52 that includes the question sentence 71 and has compressed information 73 added, which is obtained by compressing the reference information 72.

[0037] Specifically, the gatekeeper unit 110 acquires the second output information 512, which has been compressed by reducing the number of tokens in the first prompt 51, as the second prompt 52. More specifically, the gatekeeper unit 110 acquires the second prompt 52 to which compressed information 73, which has been compressed by reducing the number of tokens in the reference information 72, is added. Reducing the number of tokens leads to a reduction in the usage fees for language models such as generative AI.

[0038] The first prompt 51 may include a method for compressing the reference information 72. For example, via the input interface 930, the user may include a method for compressing the information in the first prompt 51. As a result, the gatekeeper unit 110 obtains a second prompt 52 as second output information 512, to which compressed information 73 obtained by compressing the reference information 72 using the method included in the first prompt 51 is added.

[0039] Alternatively, the gatekeeper unit 110 may have a function to add an instruction statement to the first prompt 51 that specifies the method of information compression of the reference information 72 based on the reference information 72. The gatekeeper unit 110 may also have rule information for determining the method of information compression based on the reference information 72. In this case, the gatekeeper unit 110 adds an instruction statement to the first prompt that specifies the method of information compression of the reference information 72 using the rule information. As a result, the gatekeeper unit 110 obtains a second prompt 52 as second output information 512, to which compressed information 73 obtained by compressing reference information 72 according to the instruction is added. Specific examples of information compression methods are described below.

[0040] <Specific example of information compression 1> Figure 4 is a schematic diagram showing a specific example of information compression by the language model utilization device 100 according to this embodiment. In the first concrete example of information compression, we will explain an information compression method that reduces the number of tokens in the first prompt 51 by translating the language of the first prompt 51 into a different language. For example, there is an information compression method that compresses the number of tokens by translating the language of the first prompt 51 into English.

[0041] The gatekeeper unit 110 obtains a second prompt 52 from the second language model 62, to which compressed information 73 has been added, obtained by translating the language of the reference information 72 into English. For example, the user may include an instruction in the first prompt 51 that specifies an information compression method such as "translate Japanese into English." Alternatively, if the reference information 72 is a document written in Japanese, the gatekeeper unit 110 may add an instruction to the first prompt 51 indicating an information compression method such as "translate Japanese into English."

[0042] <Specific example of information compression 2> Figure 5 is a schematic diagram showing a specific example 2 of information compression by the language model utilization device 100 according to this embodiment. In the second specific example of information compression, we will explain a method of information compression that involves shaping the reference information 72. Shaping the reference information 72 refers to information compression techniques that involve extracting necessary elements through methods such as summarization or vector search.

[0043] The gatekeeper unit 110 extracts the information used to answer the first prompt 51 from the first prompt 51 and obtains compressed second output information 512 by converting the format of the information used to answer the first prompt 51. For example, an information compression technique may be used to extract the information used to answer the first prompt 51 and convert it into tabular data. Alternatively, an information compression technique may be used to extract the information used to answer the first prompt 51 and convert it into JSON format. JSON is an abbreviation for JavaScript Object Notation. Specifically, it is as follows:

[0044] The gatekeeper unit 110 obtains a second prompt 52 to which compressed information 73 has been added, obtained by converting the information used to answer the question 71 from the reference information 72 into table data. Alternatively, the gatekeeper unit 110 obtains a second prompt 52 to which compressed information 73 has been added, obtained by converting the information used to answer the question 71 from the reference information 72 into JSON format.

[0045] The gatekeeper unit 110 may add an instruction to the first prompt 51 that instructs the search for information to be used in answering question 71 from reference information 72 and to compress that information. In this case, the gatekeeper unit 110 obtains a second prompt 52 to which compressed information 73, containing the information to be used in answering question 71, is added. Furthermore, the user may include in the first prompt 51 an instruction that "instructs the search for information to be used in answering question 71 from reference information 72 and to compress the information."

[0046] In the example in Figure 5, the first prompt 51 explicitly states the task to be solved in the reference information 72 and includes instructions to search for the work elements and necessary information required to complete the task. In this case, the first prompt 51 may include a method for compressing the information necessary to complete the task. Specifically, the first prompt 51 may include a method for indexing the information necessary to complete the task. For example, it may include a method for compressing the information necessary to complete the task by converting it into JSON format. The gatekeeper unit 110 then obtains a second prompt 52 to which compressed information 73 is attached, which consists of work elements for task completion and an index (for example, in JSON format) of information necessary for task completion.

[0047] <Answer generation process> In step S103, the response generation unit 120 inputs the second prompt 52 to the first language model 61 and obtains the first output information 521 output from the first language model 61 as the response 53 to the first prompt 51. Specifically, the response generation unit 120 inputs the second prompt 52 to the first language model 61 via the communication device 950. The first language model 61 receives the second prompt 52, outputs first output information 521 corresponding to the second prompt 52, and transmits it to the response generation unit 120 via the communication device 950. The response generation unit 120 receives the first output information 521 via the communication device 950 and stores it in the storage unit 150 as the response 53. Alternatively, the response generation unit 120 presents the response 53 to the first prompt 51 to the user via the output interface 940.

[0048] ***Other configurations*** <Example 1> The language model utilization device 100 may be configured to utilize multiple types of second language models as the second language model 62. Examples of multiple types of second language models include three types of second language models: Claude, Gemini, and CodeLLama.

[0049] At this time, the gatekeeper unit 110 inputs the first prompt 51 to each of the multiple types of second language models 62. The gatekeeper unit 110 obtains multiple second output information 512 output from each of the multiple types of second language models 62. Then, the gatekeeper unit 110 obtains one second prompt 52 from the multiple second output information 512 by ensemble (majority vote). Thus, in the language model utilization device 100 according to Modification 1, multiple types of second language models, each with its own individual function, are ensembled together. Then, in the language model utilization device 100 according to Modification 1, the output obtained from the ensemble is passed to the first language model as a second prompt 52. This makes it possible to improve the document quality of the second prompt 52.

[0050] <Modification 2> In the above description of this embodiment, the method of compressing information for reference information 72, i.e., a large amount of original documents, was described. However, all data included in the first prompt 51 may also be subject to information compression.

[0051] In the gatekeeper processing, the gatekeeper unit 110 obtains a second prompt 52, which is obtained by compressing the data contained in the first prompt 51. For example, the gatekeeper unit 110 obtains a second prompt 52 that has been compressed by reducing the number of tokens in the data contained in the first prompt 51.

[0052] Specifically, the gatekeeper unit 110 obtains a second prompt 52 by translating the language of the data contained in the first prompt 51 into another language (for example, English) to reduce the number of tokens.

[0053] In another specific example, the gatekeeper unit 110 extracts information from the data contained in the first prompt 51 to be used as a response to the first prompt 51, and obtains a compressed second prompt 52 by converting the format of the extracted information. The gatekeeper unit 110 appends an instruction to the first prompt 51 that instructs the search and extraction of information to be used as the answer to the first prompt 51. Then, the gatekeeper unit 110 obtains a compressed second prompt 52 by converting the format of the extracted information. Alternatively, the first prompt 51 may include an information compression method. The gatekeeper unit 110 may obtain a second prompt 52 obtained by compressing the data contained in the first prompt using the information compression method. Alternatively, the gatekeeper unit 110 may add an instruction statement to the first prompt 51 indicating the information compression method.

[0054] As described above, according to this embodiment, an approach may be adopted in which information compression is performed only on the large amount of source documents included in or added to the first prompt 51. Alternatively, an approach may be adopted in which information compression is performed on all the data included in the first prompt 51.

[0055] <Variation 3> In this embodiment, the functions of the gatekeeper unit 110 and the answer generation unit 120 are implemented in software. As a modification, the functions of the gatekeeper unit 110 and the answer generation unit 120 may be implemented in hardware. Specifically, the language model utilization device 100 includes an electronic circuit 909 in place of the processor 910.

[0056] Figure 6 shows an example of the configuration of a language model utilization device 100 according to a modified example of this embodiment. The electronic circuit 909 is a dedicated electronic circuit that realizes the functions of the gatekeeper unit 110 and the answer generation unit 120. Specifically, the electronic circuit 909 is a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, a logic IC, a GA, an ASIC, or an FPGA. GA is an abbreviation for Gate Array. ASIC is an abbreviation for Application Specific Integrated Circuit. FPGA is an abbreviation for Field-Programmable Gate Array.

[0057] The functions of the gatekeeper unit 110 and the answer generation unit 120 may be implemented in a single electronic circuit, or they may be implemented in a distributed manner across multiple electronic circuits.

[0058] As another variation, some functions of the gatekeeper unit 110 and the answer generation unit 120 may be implemented by electronic circuits, and the remaining functions by software. Alternatively, some or all functions of the gatekeeper unit 110 and the answer generation unit 120 may be implemented by firmware.

[0059] The processor and electronic circuit are also called processing circuits. In other words, the functions of the gatekeeper unit 110 and the response generation unit 120 are realized by the processing circuits.

[0060] ***Description of the effects of this embodiment*** In the language model utilization device according to this embodiment, the gatekeeper unit inputs a first prompt to a small-scale second language model, which is a local LLM, compresses the amount of information in the first prompt, and obtains it as a second prompt. The response generation unit inputs the second prompt to a large-scale first language model, which is a large-scale LLM, and obtains the output from the first language model as the response to the first prompt. Thus, the language model utilization device according to this embodiment can reduce the amount of prompt information input to the large-scale first language model on the cloud.

[0061] The language model utilization device according to this embodiment can be said to improve the quality of information by performing processes such as summarizing, preprocessing, or reducing the amount of data charged on the second prompt, which is the information passed to the first language model. For example, the second language model summarizes the necessary content from a large amount of input documents, compresses the amount of information, and passes the compressed result to the first language model in the cloud. Another example is extracting text from example images such as specifications, summarizing the extracted content using the second language model, and then using that summarized content to run the image's CoT. CoT is an abbreviation for Chain of Thought. Furthermore, the above example also includes improving document quality by ensembling, or majority-vote, second language models, each with its own individual functions, and passing their output to the first language model.

[0062] A large-scale first language model may be an expensive language model with high functionality and pre-programmed knowledge. For example, an expensive first language model with high functionality and pre-programmed knowledge may contain confidential knowledge or be customized. The language model utilization device according to this embodiment can reduce the amount of prompt information required for a large-scale first language model, thereby reducing the billing costs for an expensive first language model.

[0063] It is generally known that inputting a large number of user prompts into a large language model located in the cloud can result in substantial charges. On the other hand, in the language model utilization device according to this embodiment, a small language model that can run on a local PC, such as CodeLLama, can be placed in the on-premises area as the second language model. PC is an abbreviation for Personal Computer. In this case, even if a large number of prompts are input into the second language model as the first prompt, no communication occurs, and therefore no charges are incurred.

[0064] Embodiment 2. This embodiment will primarily describe the differences from Embodiment 1 and the additions made to Embodiment 1. In this embodiment, components having the same function as those in Embodiment 1 are denoted by the same reference numerals, and their descriptions are omitted. In this embodiment of the language model utilization device 100, a method for preventing adversarial prompts from being passed to the first language model 61 will be described.

[0065] ***Explanation of the structure*** The configuration of the language model utilization system 500 and the language model utilization device 100 according to this embodiment is the same as in Embodiment 1. In this embodiment, the second language model 62 includes dummy knowledge for checking whether prompt injection is being performed.

[0066] ***Explanation of operation*** Figure 7 is a flowchart showing an example of language model utilization processing by the language model utilization device 100 according to this embodiment. Figure 7 shows the flowchart from Figure 2 with steps S102a and S102b added. The other steps are the same as in Figure 2.

[0067] In step S102a, the gatekeeper unit 110 determines whether or not the second output information 512 contains dummy information. If dummy knowledge is included, proceed to step S102b. If no dummy knowledge is included, proceed to step S103. In step S102b, the gatekeeper unit 110 discards the second output information 512 and terminates processing. The gatekeeper unit 110 may also notify the user via the output interface 940 of a warning indicating that prompt injection has been performed.

[0068] As described above, the gatekeeper unit 110 inputs the first prompt 51 to the second language model 62 and checks from the output second output information 512 whether prompt injection has been performed. Specifically, the gatekeeper unit 110 determines whether injection has occurred by comparing the second output information 512 from the second language model 62 with dummy knowledge. If the determination shows that injection has not occurred, the gatekeeper unit 110 passes the second output information 512 as a second prompt to the answer generation unit 120. On the other hand, if injection has occurred, it rejects either the first prompt 51, which is user input, or the second output information 512. For example, if the second language model 62 is an LLM for RAG, the second language model 62 will examine the results to determine if there is a high injection risk, and will only allow those with a low injection risk to pass through. The second language model 62 will not accept those with a high injection risk, i.e., it will return a result of Reject. In some cases, the gatekeeper unit 110 may also check whether the output result of the LLM for RAG has been injected.

[0069] ***Other configurations*** <Modification 4> The gatekeeper unit 110 may use an external prompt check function to determine whether prompt injection has been performed at the first prompt 51. If the gatekeeper unit 110 determines that prompt injection has been performed, it performs a process to discard the first prompt 51. Thus, a configuration in which an external mechanism for checking user input is provided is also acceptable. In this case, the language model utilization device may reject high-risk first prompts based on the external checking mechanism, rather than relying on the second language model's judgment. Such a configuration is suitable when the judgment by the second language model is unstable.

[0070] <Modification 5> Alternatively, the gatekeeper unit 110 may acquire the first prompt 51 and determine whether input validation, i.e., injection, is being performed using a specific keyword. Furthermore, the specific keyword may not only be a predetermined keyword for input validation, but the content of the logic may also be dynamically changed. For example, RAG may be used to determine high-risk contexts depending on the situation.

[0071] ***Description of the effects of this embodiment*** As described above, the language model utilization device according to this embodiment can prevent adversarial prompts. Therefore, the language model utilization device according to this embodiment can prevent confidential knowledge within the LLM from being leaked to the outside due to prompt injection.

[0072] In embodiments 1 and 2 described above, each part of the language model utilization device was described as an independent functional block. However, the configuration of the language model utilization device does not have to be as described in the embodiments above. The functional blocks of the language model utilization device can be configured in any way as long as they can realize the functions described in the embodiments above. Furthermore, the language model utilization device does not have to be a single device, but may be a system composed of multiple devices. Furthermore, multiple parts of Embodiments 1 and 2 may be combined and implemented. Alternatively, only one part of these embodiments may be implemented. In addition, these embodiments may be combined and implemented in any way, either as a whole or in part. In other words, in Embodiments 1 and 2, it is possible to freely combine each embodiment, modify any component of each embodiment, or omit any component in each embodiment.

[0073] The embodiments described above are essentially preferred examples and are not intended to limit the scope of the Disclosure, the scope of the Applications of the Disclosure, or the scope of Uses of the Disclosure. The embodiments described above can be modified in various ways as needed. For example, the procedures described using flowcharts or sequence diagrams may be modified as appropriate.

[0074] The various aspects of this disclosure are summarized below as an appendix.

[0075] (Note 1) In a language model utilization device that utilizes a first language model and a second language model smaller in scale than the first language model, A gatekeeper unit that acquires a first prompt, inputs the first prompt to the second language model, and acquires the second output information, which is information output from the second language model, as a second prompt obtained by information compression of the first prompt. A response generation unit inputs the second prompt to the first language model and obtains the first output information output from the first language model as a response to the first prompt. A language model utilization device equipped with the following features. (Note 2) The gate keeper section is, The language model utilization device described in Appendix 1, which acquires the second output information, from which the first prompt has been compressed by reducing the number of tokens in the first prompt, as the second prompt. (Note 3) The gate keeper section is, A language model utilization device according to Appendix 1 or Appendix 2, which acquires the second output information obtained by reducing the number of tokens in the first prompt by translating the language of the first prompt into a language other than the language of the first prompt, as the second prompt. (Note 4) The gate keeper section is, The language model utilization device described in Appendix 1, which extracts information used to respond to the first prompt from the first prompt and obtains the second output information, which has been compressed by converting the format of the information used to respond to the first prompt, as the second prompt. (Note 5) The gate keeper section is, The language model utilization device described in Appendix 4, which appends an instruction to the first prompt that instructs the retrieval of information to be used as a response to the first prompt, and obtains the second output information, which is information-compressed, as the second prompt, from the first prompt. (Note 6) The second language model includes dummy insights to check whether prompt injection is occurring. The gate keeper section is, A language model utilization device according to any one of the appendices 1 to 5, which determines whether the second output information contains the dummy knowledge, and discards the second output information if it contains the dummy knowledge. (Note 7) The gate keeper section is, A language model utilization device according to any one of the appendices 1 to 6, which uses an external prompt checking function to determine whether prompt injection has occurred in the first prompt, and discards the first prompt if it is determined that prompt injection has occurred. (Note 8) The first prompt includes an information compression method, The gate keeper section is, A language model utilization device according to any one of the appendices 1 to 7, which acquires the second output information obtained by compressing the first prompt using the information compression method included in the first prompt, as the second prompt. (Note 9) The gate keeper section is, A language model utilization device according to any one of the appendices 1 to 7, wherein an instruction statement indicating the method of information compression is added to the first prompt, and the second output information obtained by information compression of the first prompt according to the instruction statement is acquired as the second prompt. (Note 10) The language model utilization device utilizes multiple types of second language models as the second language model, The gate keeper section is, A language model utilization device according to any one of the appendices 1 to 9, which inputs the first prompt to each of the plurality of second language models, obtains a plurality of second output information outputs from each of the plurality of second language models, and obtains one second prompt from the plurality of second output information by majority vote. (Note 11) Each of the first language model and the second language model is a generative AI (Artificial Intelligence) language model utilization device as described in any one of the appendices 1 to 10. (Note 12) The first prompt includes a question and is accompanied by reference information to be used in answering the question. The gate keeper section is, The first prompt is obtained, the first prompt is input to the second language model, and the second output information, which is information output from the second language model, is obtained as a second prompt that includes the question text and has compressed information obtained by compressing the reference information added to it. The aforementioned response generation unit, A language model utilization device according to Appendix 1 or Appendix 2, which inputs the second prompt to the first language model and obtains the first output information output from the first language model as a response to the first prompt. (Note 13) In a language model utilization method used in a language model utilization device that utilizes a first language model and a second language model smaller in scale than the first language model, The computer obtains a first prompt, inputs the first prompt into the second language model, and obtains the second output information, which is information output from the second language model, as a second prompt obtained by information compression of the first prompt. A method for utilizing a language model, wherein a computer inputs the second prompt to the first language model and obtains the first output information output from the first language model as a response to the first prompt. (Note 14) In a language model utilization program used in a language model utilization device that utilizes a first language model and a second language model smaller in scale than the first language model, A gatekeeper process that obtains a first prompt, inputs the first prompt to the second language model, and obtains the second output information, which is the information output from the second language model, as a second prompt obtained by information compression of the first prompt. A response generation process that inputs the second prompt to the first language model and obtains the first output information output from the first language model as the response to the first prompt. A language model-based program that allows a computer to execute a command. [Explanation of Symbols]

[0076] 51 First prompt, 512 Second output information, 52 Second prompt, 521 First output information, 53 Answer, 61 First language model, 62 Second language model, 71 Question text, 72 Reference information, 73 Compressed information, 100 Language model utilization device, 110 Gatekeeper unit, 120 Answer generation unit, 150 Memory unit, 200 Artificial intelligence, 500 Language model utilization system, 909 Electronic circuit, 910 Processor, 921 Memory, 922 Auxiliary storage device, 930 Input interface, 940 Output interface, 950 Communication device.

Claims

1. In a language model utilization device that utilizes a first language model and a second language model smaller in scale than the first language model, A gatekeeper unit that acquires a first prompt, inputs the first prompt to the second language model, and acquires the second output information, which is information output from the second language model, as a second prompt obtained by information compression of the first prompt. A response generation unit inputs the second prompt to the first language model and obtains the first output information output from the first language model as a response to the first prompt. A language model utilization device equipped with the following features.

2. The gate keeper section is, The language model utilization device according to claim 1, wherein the second output information, from which the first prompt has been compressed by reducing the number of tokens in the first prompt, is acquired as the second prompt.

3. The gate keeper section is, The language model utilization device according to claim 1 or claim 2, wherein the second output information obtained by reducing the number of tokens in the first prompt by translating the language of the first prompt into a language other than the language of the first prompt is acquired as the second prompt.

4. The gate keeper section is, The language model utilization device according to claim 1, which extracts information used to respond to the first prompt from the first prompt and obtains the second output information, which has been compressed by converting the format of the information used to respond to the first prompt, as the second prompt.

5. The gate keeper section is, The language model utilization device according to claim 4, wherein an instruction statement instructing the retrieval of information to be used as a response to the first prompt is added to the first prompt, and the second output information, in which the information to be used as a response to the first prompt has been compressed, is obtained as the second prompt.

6. The second language model includes dummy insights to check whether prompt injection is occurring. The gate keeper section is, A language model utilization device according to claim 1 or claim 2, which determines whether the second output information contains the dummy knowledge, and discards the second output information if it contains the dummy knowledge.

7. The gate keeper section is, A language model utilization device according to claim 1 or 2, which uses an external prompt check function to determine whether prompt injection has occurred in the first prompt, and discards the first prompt if it is determined that prompt injection has occurred.

8. The first prompt includes an information compression method, The gate keeper section is, The language model utilization device according to claim 1 or claim 2, wherein the second output information obtained by compressing the first prompt using the information compression method included in the first prompt is acquired as the second prompt.

9. The gate keeper section is, The language model utilization device according to claim 1 or claim 2, wherein an instruction statement indicating an information compression method is added to the first prompt, and the second output information obtained by information compression of the first prompt according to the instruction statement is acquired as the second prompt.

10. The language model utilization device utilizes multiple types of second language models as the second language model, The gate keeper section is, A language model utilization device according to claim 1 or claim 2, which inputs the first prompt to each of the plurality of types of second language models, obtains a plurality of second output information outputs from each of the plurality of types of second language models, and obtains one second prompt from the plurality of second output information by majority vote.

11. The language model utilization device according to claim 1 or claim 2, wherein each of the first language model and the second language model is a generative AI (Artificial Intelligence).

12. The first prompt includes a question and is accompanied by reference information to be used in answering the question. The gate keeper section is, The first prompt is obtained, the first prompt is input to the second language model, and the second output information, which is information output from the second language model, is obtained as a second prompt that includes the question text and has compressed information obtained by compressing the reference information added to it. The aforementioned response generation unit, A language model utilization device according to claim 1 or claim 2, wherein the second prompt is input to the first language model, and the first output information output from the first language model is obtained as a response to the first prompt.

13. In a language model utilization method used in a language model utilization device that utilizes a first language model and a second language model smaller in scale than the first language model, The computer acquires a first prompt, inputs the first prompt into the second language model, and acquires the second output information, which is information output from the second language model, as a second prompt obtained by information compression of the first prompt. A method for utilizing a language model, wherein a computer inputs the second prompt to the first language model and obtains the first output information output from the first language model as a response to the first prompt.

14. In a language model utilization program used in a language model utilization device that utilizes a first language model and a second language model smaller in scale than the first language model, A gatekeeper process that obtains a first prompt, inputs the first prompt to the second language model, and obtains the second output information, which is information output from the second language model, as a second prompt obtained by information compression of the first prompt. A response generation process that inputs the second prompt to the first language model and obtains the first output information output from the first language model as a response to the first prompt. A language model-based program that allows a computer to execute a command.

Images