Language model utilization device, language model utilization method, and language model utilization program

Abstract

The purpose is to reduce the amount of information in the prompt, lower the utilization rate of large-scale language models, and reduce the costs involved in using large-scale language models. [Solution] A language model utilization device (100) utilizes a first language model (61) and a second language model (62) that is smaller in scale than the first language model. A gatekeeper unit (110) inputs a first prompt (51) to the second language model (62) and acquires second output information output from the second language model (62). The gatekeeper unit (110) acquires the second output information as a second prompt (52) obtained by compressing the first prompt (51). An answer generation unit (120) inputs the second prompt (52) to the first language model (61) and acquires the first output information output from the first language model (61) as an answer (53) to the first prompt (51).

Description

[Technical Field]

[0001] The present disclosure relates to a language model utilization device, a language model utilization method, and a language model utilization program. [Background technology]

[0002] In recent years, systems that perform natural language processing such as question-and-answering have been provided that utilize large-scale language models called LLMs. LLMs are natural language processing models trained using large amounts of text data. When LLMs are applied to a question-and-answering system, when a prompt containing a question is input into the LLM, the LLM outputs an answer to the question.

[0003] However, as the amount of information entered into LLMs increases, the cost of using LLMs increases. In particular, with cloud-based LLMs, the usage fee can become very high depending on the amount of information entered into the prompts.

[0004] Patent Document 1 discloses a technology for efficiently training an SLM to be added to an LLM so that output sentences from the SLM reflect knowledge in a specific domain. SLM is an abbreviation for Small Language Models. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 7442239 Summary of the Invention [Problem to be solved by the invention]

[0006] The technology in Patent Document 1 can reduce the amount of calculation during learning, but cannot reduce the amount of calculation during answer generation. Therefore, there is a problem that the cost of using LLM increases as the amount of information in prompts input to LLM increases.

[0007] The present disclosure aims to reduce the cost of using LLM by reducing the amount of prompt information input to LLM and lowering the utilization rate of LLM. [Means for solving the problem]

[0008] A language model utilization device according to the present disclosure includes: 1. A language model utilization device that utilizes a first language model and a second language model that is smaller in scale than the first language model, a gatekeeper unit that acquires a first prompt, inputs the first prompt into the second language model, and acquires second output information, which is information output from the second language model, as a second prompt obtained by compressing the first prompt; an answer generation unit that inputs the second prompt into the first language model and acquires first output information output from the first language model as an answer to the first prompt; Equipped with. [Effects of the Invention]

[0009] A language model utilization device according to the present disclosure utilizes a first language model and a second language model that is smaller in scale than the first language model. The language model utilization device according to the present disclosure uses the second language model to reduce the amount of prompt information input to the first language model. Therefore, the language model utilization device according to the present disclosure has the effect of reducing the utilization rate of the first language model and reducing the cost of using the first language model. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a language model utilization device according to a first embodiment. [Figure 2] FIG. 1 is a diagram showing an example of the functional configuration of a language model utilization system according to a first embodiment. [Figure 3] 4 is a flowchart showing an example of a language model utilization process performed by the language model utilization device according to the first embodiment. [Figure 4] FIG. 2 is a schematic diagram showing a specific example 1 of information compression performed by the language model utilization device according to the first embodiment. [Figure 5] FIG. 4 is a schematic diagram showing a specific example 2 of information compression performed by the language model utilization device according to the first embodiment. [Figure 6] FIG. 10 is a diagram showing an example of the configuration of a language model utilization device according to a modification of the first embodiment. [Figure 7] FIG. 10 is a flowchart showing an example of a language model utilization process performed by a language model utilization device according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] The present embodiment will be described below with reference to the drawings. In each drawing, the same or corresponding parts are assigned the same reference numerals. In the description of the embodiment, the description of the same or corresponding parts will be omitted or simplified as appropriate. The arrows in the drawings mainly indicate the flow of data or the flow of processing.

[0012] Embodiment 1 ***Configuration Description*** FIG. 1 is a diagram showing an example of the configuration of a language model utilization device 100 according to this embodiment. FIG. 2 is a diagram showing an example of the functional configuration of a language model utilization system 500 according to this embodiment. The language model utilization device 100 is a computer. The language model utilization device 100 utilizes a first language model 61 and a second language model 62 that is smaller in scale than the first language model 61. The language model utilization device 100 is, for example, a question and answer system that uses the first language model 61 and the second language model 62.

[0013] The language model utilization system 500 includes a language model utilization device 100 and a first language model 61 that communicates with the language model utilization device 100. In this embodiment, the second language model 62 is provided in the language model utilization device 100. Note that the second language model 62 may also be provided outside the language model utilization device 100 so as to be able to communicate with the language model utilization device 100.

[0014] The first language model 61 is also called artificial intelligence 200. The first language model 61 is, for example, a type of generative AI specialized in natural language processing. AI is an abbreviation for Artificial Intelligence. The first language model 61 is a large-scale LLM that achieves more advanced language understanding by learning from massive amounts of text data. Specific examples of the first language model 61 include GPT-4, PaLM2, and LLaMa2. GPT is an abbreviation for Generative Pre-trained Transformer. PaLM is an abbreviation for Pathways Language Model. LLaMa is an abbreviation for Large Language Model meta AI. The first language model 61 is provided, for example, on the cloud. The usage fee for the first language model 61 increases as the amount of information in the input prompt increases. The first language model 61 is also referred to as the first LLM.

[0015] The second language model 62 is also called artificial intelligence 200, and is a type of generative AI specialized in natural language processing. In this embodiment, the second language model 62 is assumed to be a language model smaller in scale than the first language model 61. Specific examples of the second language model 62 include Claude, Gemini, and CodeLLama. The second language model 62 is provided, for example, in an on-premise environment. In this embodiment, the second language model 62 is provided inside the language model utilization device 100, but it may also be provided outside the language model utilization device 100. When the second language model 62 is provided outside the language model utilization device 100, a usage fee may be charged, but it can be used more cheaply than the first language model 61. Furthermore, when the second language model 62 is provided in an on-premise environment, it can be used inexpensively, with the only cost being the communication traffic. The second language model 62 is also referred to as a second LLM or a local LLM.

[0016] The language model utilization device 100 includes a processor 910, as well as other hardware such as a memory 921, an auxiliary storage device 922, an input interface 930, an output interface 940, and a communication device 950. The processor 910 is connected to the other hardware via signal lines and controls the other hardware.

[0017] Language model utilization device 100 includes, as functional elements, gatekeeper unit 110, answer generation unit 120, and storage unit 150. Storage unit 150 stores second language model 62, first prompt 51, second prompt 52, and answer 53.

[0018] The functions of the gatekeeper unit 110 and the answer generation unit 120 are realized by software. The storage unit 150 is provided in the memory 921. The storage unit 150 may be provided in the auxiliary storage device 922, or may be provided separately in the memory 921 and the auxiliary storage device 922.

[0019] The processor 910 is a device that executes a language model utilization program. The language model utilization program is a program that realizes the functions of the gatekeeper unit 110 and the answer generation unit 120. The processor 910 is an IC that performs arithmetic processing. Specific examples of the processor 910 are a CPU, a DSP, and a GPU. IC is an abbreviation for Integrated Circuit. CPU is an abbreviation for Central Processing Unit. DSP is an abbreviation for Digital Signal Processor. GPU is an abbreviation for Graphics Processing Unit.

[0020] The memory 921 is a storage device that temporarily stores data. Specific examples of the memory 921 are SRAM and DRAM. SRAM is an abbreviation for Static Random Access Memory. DRAM is an abbreviation for Dynamic Random Access Memory. The auxiliary storage device 922 is a storage device that stores data. A specific example of the auxiliary storage device 922 is a HDD. The auxiliary storage device 922 may also be a portable storage medium such as an SD (registered trademark) memory card, CF, NAND flash, a flexible disk, an optical disk, a compact disk, a Blu-ray (registered trademark) disk, or a DVD. Note that HDD is an abbreviation for Hard Disk Drive. SD (registered trademark) is an abbreviation for Secure Digital. CF is an abbreviation for CompactFlash (registered trademark). DVD is an abbreviation for Digital Versatile Disk.

[0021] The input interface 930 is a port connected to an input device such as a mouse, keyboard, or touch panel. Specifically, the input interface 930 is a USB terminal. The input interface 930 may also be a port connected to a LAN. USB is an abbreviation for Universal Serial Bus. LAN is an abbreviation for Local Area Network.

[0022] The output interface 940 is a port to which a cable of an output device such as a display is connected. Specifically, the output interface 940 is a USB terminal or an HDMI (registered trademark) terminal. Specifically, the display is an LCD. The output interface 940 is also called a display interface. HDMI (registered trademark) is an abbreviation for High Definition Multimedia Interface. LCD is an abbreviation for Liquid Crystal Display.

[0023] The communication device 950 has a receiver and a transmitter. The communication device 950 is connected to a communication network such as a LAN, the Internet, a telephone line, or Wi-Fi (registered trademark). Specifically, the communication device 950 is a communication chip or NIC. NIC is an abbreviation for Network Interface Card.

[0024] The language model utilization program is executed in the language model utilization device 100. The language model utilization program is read into the processor 910 and executed by the processor 910. In addition to the language model utilization program, an OS is also stored in the memory 921. OS is an abbreviation for Operating System. The processor 910 executes the language model utilization program while executing the OS. The language model utilization program and the OS may be stored in an auxiliary storage device 922. The language model utilization program and the OS stored in the auxiliary storage device 922 are loaded into the memory 921 and executed by the processor 910. Note that part or all of the language model utilization program may be incorporated into the OS.

[0025] The language model utilization device 100 may include multiple processors that replace the processor 910. These multiple processors share the task of executing the language model utilization program. Each processor is a device that executes the language model utilization program, just like the processor 910.

[0026] Data, information, signal values, and variable values ​​used, processed, or output by the language model utilization program are stored in memory 921, auxiliary storage device 922, or registers or cache memory within processor 910.

[0027] The "part" of each of the gatekeeper unit 110 and the answer generation unit 120 may be read as a "circuit," "step," "procedure," "process," or "circuitry." The language model utilization program causes a computer to execute gatekeeper processing and answer generation processing. The "processing" of the gatekeeper processing and answer generation processing may be read as a "program," "program product," "computer-readable storage medium storing a program," or "computer-readable recording medium recording a program." Furthermore, the language model utilization method is a method performed by the language model utilization device 100 executing the language model utilization program. The language model utilization program may be provided by being stored in a computer-readable recording medium, or may be provided as a program product.

[0028] As shown in FIG. 2, each of the first language model 61 and the second language model 62 is a type of artificial intelligence 200. The artificial intelligence 200 includes an inference unit that performs inference using a trained model. The inference unit receives a prompt as input and outputs an answer corresponding to the prompt as output information based on the trained model. In this embodiment, first output information is output from the first language model 61. Second output information is output from the second language model 62.

[0029] The artificial intelligence 200 is, for example, a generative AI. The artificial intelligence 200 is configured using algorithms such as Transformer, BERT, and GPT. The artificial intelligence 200 may also be configured by combining multiple algorithms including these algorithms. BERT is an abbreviation for Bidirectional Encoder Representations from Transformers.

[0030] As described above, in this embodiment, the artificial intelligence 200 uses the first language model 61 and the second language model 62, which is smaller in scale than the first language model. 2 shows a configuration in which the second language model 62 also includes an inference unit, the gatekeeper unit 110 may also function as the inference unit of the second language model 62. RAG is a suitable example of a generative AI design pattern for the second language model 62. RAG is an abbreviation for Retrieval Augmented Generation. RAG is a technology that searches for and extracts desired information and has the generative AI generate answers based on that content. This allows the generative AI to generate answers not only from learned information, but also from unlearned internal information, etc. The second language model may be an SLM, or may be composed of multiple types of language models including an SLM.

[0031] ***Explanation of Operation*** Next, the operation of the language model utilization device 100 according to this embodiment will be described. The operating procedure of the language model utilization device 100 corresponds to a language model utilization method. Furthermore, a program that realizes the language model utilization process, which is the operation of the language model utilization device 100, corresponds to a language model utilization program.

[0032] FIG. 3 is a flow diagram showing an example of a language model utilization process performed by the language model utilization device 100 according to this embodiment. The language model utilization process includes a gatekeeper process and an answer generation process. In the gatekeeper processing, the gatekeeper unit 110 acquires a first prompt 51 and inputs the first prompt 51 to a second language model 62. The gatekeeper unit 110 acquires second output information 512, which is information output from the second language model 62, as a second prompt 52 obtained by compressing the first prompt 51. In the answer generation process, the answer generation unit 120 inputs the second prompt 52 to the first language model 61 and obtains first output information 521 output from the first language model 61 as an answer 53 to the first prompt 51.

[0033] In this embodiment, an example of first prompt 51 will be described using a prompt consisting of a question 71 and reference information 72 used to answer question 71. Reference information 72 is, for example, a large amount of original documents used to answer question 71. The following description will be directed to a mode of compressing the reference information 72, i.e., a large amount of original document information. However, the question 71 and the reference information 72 may be compressed together. The mode of compressing the question 71 and the reference information 72 together will be described later.

[0034] In the gatekeeper processing, the gatekeeper unit 110 acquires a first prompt 51 including a question sentence 71. Reference information 72 used to answer the question sentence 71 is added to the first prompt 51. The gatekeeper unit 110 inputs the first prompt 51 to a second language model 62 and acquires second output information 512, which is information output from the second language model 62. The gatekeeper unit 110 acquires the second output information 512 as the second prompt 52 including the question sentence 71 to which compressed information 73 obtained by compressing the reference information 72 has been added. In the answer generation process, the answer generation unit 120 inputs the second prompt 52 to the first language model 61 via the communication device 950, and obtains the first output information 521 output from the first language model 61 as an answer 53 to the first prompt 51. Specifically, it is as follows:

[0035] <Gatekeeper processing> In step S101, gatekeeper unit 110 acquires first prompt 51 including question sentence 71. First prompt 51 has reference information 72 used to answer question sentence 71 added thereto. Specifically, the gatekeeper unit 110 receives a first prompt 51 from the user via the input interface 930 or the communication device 950 .

[0036] In step S102, the gatekeeper unit 110 inputs the first prompt 51 to the second language model 62 and acquires second output information 512 output from the second language model 62. The second output information 512 is the second prompt 52 including the question sentence 71, to which compressed information 73 obtained by compressing the reference information 72 has been added. The gatekeeper unit 110 acquires the second output information 512 as the second prompt 52 including the question sentence 71 and to which compressed information 73 obtained by compressing the reference information 72 has been added.

[0037] Specifically, the gatekeeper unit 110 acquires second output information 512, which has been compressed by reducing the number of tokens in the first prompt 51, as the second prompt 52. More specifically, the gatekeeper unit 110 acquires the second prompt 52 to which compressed information 73, which has been compressed by reducing the number of tokens in the reference information 72, has been added. Reducing the number of tokens leads to a reduction in the usage fee for a language model such as a generative AI.

[0038] The first prompt 51 may include a method for compressing the reference information 72. For example, the user may include the method for compressing the information in the first prompt 51 via the input interface 930. As a result, the gatekeeper unit 110 obtains, as the second output information 512, the second prompt 52 to which compressed information 73 obtained by compressing the reference information 72 using the method included in the first prompt 51 has been added.

[0039] Alternatively, the gatekeeper unit 110 may have a function of adding an instruction statement instructing the information compression method of the reference information 72 to the first prompt 51 based on the reference information 72. The gatekeeper unit 110 may have rule information for determining the information compression method based on the reference information 72. In this case, the gatekeeper unit 110 adds an instruction statement instructing the information compression method of the reference information 72 to the first prompt using the rule information. As a result, the gatekeeper unit 110 acquires, as second output information 512, the second prompt 52 to which compressed information 73 obtained by compressing the reference information 72 in accordance with the instruction sentence has been added. A specific example of the information compression method will be explained below.

[0040] <Example of data compression 1> FIG. 4 is a schematic diagram showing a specific example 1 of information compression by the language model utilization device 100 according to this embodiment. In specific example 1 of information compression, a method of information compression will be described in which the number of tokens in first prompt 51 is reduced by translating the language of first prompt 51 into a language different from that of first prompt 51. For example, there is a method of information compression in which the number of tokens is compressed by translating the language of first prompt 51 into English.

[0041] The gatekeeper unit 110 obtains from the second language model 62 the second prompt 52 to which the compressed information 73, which is information compressed by translating the language of the reference information 72 into English, has been added. For example, the user may include in first prompt 51 an instruction statement instructing the method of information compression, such as "translate Japanese into English." Alternatively, if the reference information 72 is a document written in Japanese, the gatekeeper unit 110 may add to the first prompt 51 an instruction statement instructing the method of information compression, such as "translate Japanese into English."

[0042] <Example of data compression 2> FIG. 5 is a schematic diagram showing a specific example 2 of information compression by the language model utilization device 100 according to this embodiment. In the second specific example of information compression, we will explain a method of compressing information by shaping the reference information 72. Shaping the reference information 72 is an information compression method in which necessary elements are extracted using techniques such as summarization or vector search.

[0043] The gatekeeper unit 110 extracts information from the first prompt 51 that is to be used in response to the first prompt 51, and obtains compressed second output information 512 by converting the format of the information that is to be used in response to the first prompt 51. For example, an information compression technique may be used in which information to be used in response to the first prompt 51 is extracted and converted into table data. Alternatively, an information compression technique may be used in which information to be used in response to the first prompt 51 is extracted and converted into JSON format. JSON is an abbreviation for JavaScript Object Notation. Specifically, it is as follows:

[0044] The gatekeeper unit 110 acquires the second prompt 52 to which the compressed information 73, which is information used to answer the question 71 among the reference information 72, has been added by converting the information into table data. Alternatively, the gatekeeper unit 110 acquires the second prompt 52 to which compressed information 73, which is information used to answer the question 71 among the reference information 72, has been added by converting the information into JSON format.

[0045] The gatekeeper unit 110 may add "an instruction to search for information to be used in answering the question 71 from the reference information 72 and to compress the information" to the first prompt 51. At this time, the gatekeeper unit 110 acquires the second prompt 52 to which the compressed information 73, which is information to be used in answering the question 71, has been added. Furthermore, the first prompt 51 may include an instruction to search for information to be used in answering the question 71 from the reference information 72 and compress the information.

[0046] In the example of FIG. 5, the first prompt 51 clearly indicates the task that the user really wants to solve in the reference information 72, and includes instructions for searching for the work elements and necessary information required to complete the task. In this case, the first prompt 51 may also include a method for compressing the information required to complete the task. Specifically, the first prompt 51 may include a method for indexing the information required to complete the task. For example, the first prompt 51 may include a method for compressing the information required to complete the task, such as converting the information required to complete the task into JSON format. Then, the gatekeeper unit 110 acquires the second prompt 52 to which the compressed information 73, which is made up of the work elements up to the completion of the task and an index of the information required to complete the task (for example, in JSON format), has been added.

[0047] <Answer generation process> In step S103, the answer generation unit 120 inputs the second prompt 52 to the first language model 61, and obtains the first output information 521 output from the first language model 61 as the answer 53 to the first prompt 51. Specifically, the answer generation unit 120 inputs the second prompt 52 to the first language model 61 via the communication device 950. The first language model 61 acquires the second prompt 52, outputs first output information 521 in accordance with the second prompt 52, and transmits the first output information 521 to the answer generation unit 120 via the communication device 950. The answer generation unit 120 receives the first output information 521 via the communication device 950 and stores it in the storage unit 150 as an answer 53. Alternatively, the answer generation unit 120 presents the answer 53 to the first prompt 51 to the user via the output interface 940.

[0048] ***Other Configurations*** <Variation 1> The language model utilization device 100 may be configured to utilize a plurality of types of second language models as the second language model 62. Examples of the plurality of types of second language models include three types of second language models: Claude, Gemini, and CodeLLama.

[0049] At this time, the gatekeeper unit 110 inputs the first prompt 51 to each of the multiple types of second language models 62. The gatekeeper unit 110 acquires multiple pieces of second output information 512 output from each of the multiple types of second language models 62. Then, the gatekeeper unit 110 acquires one second prompt 52 from the multiple pieces of second output information 512 by ensemble (majority vote). In this way, the language model utilization device 100 according to the first modification ensembles multiple types of second language models, each having its own individual function. Then, the language model utilization device 100 according to the first modification 1 passes the output obtained by the ensemble to the first language model as the second prompt 52. This makes it possible to improve the document quality of the second prompt 52.

[0050] <Variation 2> In the above description of this embodiment, the information compression is performed on the reference information 72, i.e., a large amount of original document. However, all data included in the first prompt 51 may be subject to information compression.

[0051] In the gatekeeper process, gatekeeper unit 110 obtains second prompt 52 obtained by compressing the data included in first prompt 51 . For example, gatekeeper unit 110 acquires second prompt 52, which is information-compressed by reducing the number of data tokens included in first prompt 51.

[0052] Specifically, gatekeeper unit 110 obtains second prompt 52 with a reduced number of tokens by translating the language of the data included in first prompt 51 into another language (for example, English).

[0053] In another specific example, the gatekeeper unit 110 extracts information to be used in answering the first prompt 51 from the data contained in the first prompt 51, and obtains the second prompt 52 with compressed information by converting the format of the extracted information. Gatekeeper unit 110 appends to first prompt 51 an instruction statement that instructs searching and extracting information to be used in answering first prompt 51. Gatekeeper unit 110 then converts the format of the extracted information to obtain second prompt 52, which is information-compressed. Alternatively, the first prompt 51 may include an information compression method. The gatekeeper unit 110 may obtain the second prompt 52 by compressing the data included in the first prompt using the information compression method. Alternatively, the gatekeeper unit 110 may add an instruction to the first prompt 51 to specify the information compression method.

[0054] As described above, according to this embodiment, it is possible to adopt a mode in which information is compressed only for a large amount of original document included in or added to the first prompt 51. It is also possible to adopt a mode in which information is compressed for all data included in the first prompt 51.

[0055] <Variation 3> In this embodiment, the functions of gatekeeper section 110 and answer generation section 120 are realized by software. As a modification, the functions of gatekeeper section 110 and answer generation section 120 may be realized by hardware. Specifically, the language model utilization device 100 includes an electronic circuit 909 instead of a processor 910 .

[0056] FIG. 6 is a diagram showing an example of the configuration of a language model utilization device 100 according to a modified example of this embodiment. The electronic circuit 909 is a dedicated electronic circuit that realizes the functions of the gatekeeper unit 110 and the answer generation unit 120. Specifically, the electronic circuit 909 is a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, a logic IC, a GA, an ASIC, or an FPGA. GA is an abbreviation for Gate Array. ASIC is an abbreviation for Application Specific Integrated Circuit. FPGA is an abbreviation for Field-Programmable Gate Array.

[0057] The functions of the gatekeeper section 110 and the answer generating section 120 may be realized by one electronic circuit, or may be realized by distributing them among a plurality of electronic circuits.

[0058] As another variation, some of the functions of gatekeeper unit 110 and answer generation unit 120 may be realized by electronic circuits, with the remaining functions realized by software. Also, some or all of the functions of gatekeeper unit 110 and answer generation unit 120 may be realized by firmware.

[0059] Each of the processor and the electronic circuit is also called processing circuitry. That is, the functions of the gatekeeper unit 110 and the answer generation unit 120 are realized by the processing circuitry.

[0060] ***Explanation of the effect of this embodiment*** In the language model utilization device according to this embodiment, the gatekeeper unit inputs a first prompt to a small-scale second language model, which is a local LLM, compresses the information content of the first prompt, and acquires it as a second prompt. The answer generation unit inputs the second prompt to a first language model, which is a large-scale LLM, and acquires the output from the first language model as an answer to the first prompt. In this way, the language model utilization device according to this embodiment can reduce the amount of information in the prompts to be input to a large-scale first language model on the cloud.

[0061] The language model utilization device according to this embodiment can be said to improve the quality of the amount of information by summarizing, preprocessing, or reducing charges for the second prompt, which is information passed to the first language model. For example, the second language model summarizes necessary content from a large amount of input documents, compresses the amount of information, and passes the compressed results to the first language model in the cloud. As another example, characters are extracted from a sample image such as a specification document, and the extracted content is summarized in the second language model. The summarized content is then used to cycle the CoT of the image. CoT is an abbreviation for Chain of Thought. The above example also includes improving the quality of documents by ensembling second language models, each with its own individual features, i.e., by having them vote, and then passing the output to the first language model.

[0062] The large-scale first language model may be an expensive language model with advanced knowledge input. For example, the expensive first language model with advanced knowledge input may contain confidential knowledge or may be customized. The language model utilization device according to this embodiment can reduce the amount of information prompting the large-scale first language model, thereby reducing the billing costs for the expensive first language model.

[0063] It is generally known that inputting a large number of prompts from a user into a large-scale language model on the cloud can result in a significant charge. On the other hand, in a language model utilization device according to this embodiment, a small-scale language model, such as CodeLLama, that can run on a local PC or the like can be deployed in an on-premise area as the second language model. PC is an abbreviation for personal computer. In this case, even if a large number of prompts are input into the second language model as first prompts, no communication occurs, which has the advantage of not incurring a charge.

[0064] Embodiment 2 In this embodiment, differences from and additions to the first embodiment will be mainly described. In this embodiment, components having the same functions as those in the first embodiment are given the same reference numerals, and the description thereof will be omitted. In the language model utilization device 100 according to this embodiment, a mode in which hostile prompts are not passed through the first language model 61 will be described.

[0065] ***Configuration Description*** The configurations of the language model utilization system 500 and the language model utilization device 100 according to this embodiment are the same as those in the first embodiment. In this embodiment, the second language model 62 includes dummy knowledge for checking whether prompt injection is being performed.

[0066] ***Explanation of Operation*** FIG. 7 is a flow diagram showing an example of a language model utilization process performed by the language model utilization device 100 according to this embodiment. In Fig. 7, steps S102a and S102b are added to the flow diagram of Fig. 2. The other steps are the same as those in Fig. 2.

[0067] In step S102a, the gatekeeper unit 110 determines whether or not the second output information 512 includes a dummy finding. If the dummy knowledge is included, the process proceeds to step S102b. If no dummy findings are included, the process proceeds to step S103. In step S102b, the gatekeeper unit 110 discards the second output information 512 and ends the process. Note that the gatekeeper unit 110 may notify the user via the output interface 940 of a warning indicating that prompt injection has been performed.

[0068] As described above, gatekeeper unit 110 inputs first prompt 51 to second language model 62, and checks from output second output information 512 whether prompt injection has been performed. Specifically, the gatekeeper unit 110 determines whether injection has occurred by comparing the second output information 512 from the second language model 62 with the dummy knowledge. If the determination result shows that injection has not occurred, the gatekeeper unit 110 passes the second output information 512 as a second prompt to the answer generation unit 120. On the other hand, if injection has occurred, the gatekeeper unit 110 rejects the first prompt 51 or the second output information 512, which are user inputs. For example, if the second language model 62 is an LLM for RAG, the second language model 62 examines the results to determine whether there is a high risk of injection and passes only those with a low risk of injection. The second language model 62 does not accept those with a high risk of injection, i.e., returns a result of Reject. In some cases, the gatekeeper unit 110 may examine whether the output result of the LLM for RAG has been injected.

[0069] ***Other Configurations*** <Variation 4> Gatekeeper unit 110 may use an external prompt check function to determine whether prompt injection is being performed in first prompt 51. If gatekeeper unit 110 determines that prompt injection is being performed, it performs processing to discard first prompt 51. In this way, a configuration may be adopted in which an external mechanism is provided to check the user's input. In this case, the language model utilization device may not accept a first prompt with a high risk level by using the external checking mechanism, rather than by using the second language model for determination. This configuration is suitable when the determination by the second language model is unstable.

[0070] <Variation 5> Alternatively, the gatekeeper unit 110 may acquire the first prompt 51 and perform input checks using a specific keyword, i.e., determine whether injection is occurring. Furthermore, the specific keyword may not only be a predetermined keyword, but also dynamically change the logic content. For example, RAG may be used to determine high-risk contexts depending on the situation.

[0071] ***Explanation of the effect of this embodiment*** As described above, the language model utilization device according to this embodiment can prevent hostile prompts. Therefore, the language model utilization device according to this embodiment can prevent knowledge that should be kept secret within the LLM from being leaked to the outside due to prompt injection.

[0072] In the above first and second embodiments, each unit of the language model utilization device has been described as an independent functional block. However, the configuration of the language model utilization device does not have to be the same as that of the above-described embodiments. The functional blocks of the language model utilization device may have any configuration as long as they can realize the functions described in the above-described embodiments. Furthermore, the language model utilization device may not be a single device, but may be a system made up of multiple devices. Furthermore, it is possible to combine multiple parts of the first and second embodiments. Alternatively, it is possible to implement only one part of these embodiments. In addition, it is possible to implement any combination of these embodiments, either as a whole or in part. That is, in the first and second embodiments, the respective embodiments can be freely combined, or any of the components in the respective embodiments can be modified, or any of the components in the respective embodiments can be omitted.

[0073] The above-described embodiments are essentially preferred examples and are not intended to limit the scope of the present disclosure, the scope of application of the present disclosure, or the scope of use of the present disclosure. The above-described embodiments can be modified in various ways as needed. For example, the procedures described using flow charts or sequence diagrams may be modified as appropriate.

[0074] Various aspects of the present disclosure are summarized below as appendices.

[0075] (Appendix 1) 1. A language model utilization device that utilizes a first language model and a second language model that is smaller in scale than the first language model, a gatekeeper unit that acquires a first prompt, inputs the first prompt into the second language model, and acquires second output information, which is information output from the second language model, as a second prompt obtained by compressing the first prompt; an answer generation unit that inputs the second prompt into the first language model and acquires first output information output from the first language model as an answer to the first prompt; A language model utilization device comprising: (Appendix 2) The gatekeeper unit 2. The language model utilization device according to claim 1, wherein the second output information obtained by compressing the first prompt by reducing the number of tokens of the first prompt is acquired as the second prompt. (Appendix 3) The gatekeeper unit 3. The language model utilization device according to claim 1, wherein the second output information is obtained as the second prompt by translating a language of the first prompt into a language different from the language of the first prompt to reduce the number of tokens of the first prompt. (Appendix 4) The gatekeeper unit 2. The language model utilization device according to claim 1, wherein information to be used in response to the first prompt is extracted from the first prompt, and the second output information is compressed by converting a format of the information to be used in response to the first prompt, and the second output information is acquired as the second prompt. (Appendix 5) The gatekeeper unit 5. The language model utilization device according to claim 4, wherein an instruction sentence for instructing a search for information to be used in answering the first prompt is added to the first prompt, and the second output information in which information to be used in answering the first prompt is compressed is acquired as the second prompt. (Appendix 6) The second language model includes dummy knowledge for checking whether prompt injection is performed; The gatekeeper unit 6. The language model utilization device according to claim 1, wherein the second output information is determined to include the dummy knowledge, and if the dummy knowledge is included, the second output information is discarded. (Appendix 7) The gatekeeper unit 7. The language model utilization device according to claim 1, wherein an external prompt check function is used to determine whether prompt injection is being performed in the first prompt, and if it is determined that prompt injection is being performed, the first prompt is discarded. (Appendix 8) the first prompt includes a method of information compression; The gatekeeper unit 8. The language model utilization device according to claim 1, wherein the second output information obtained by compressing the first prompt using the information compression method included in the first prompt is acquired as the second prompt. (Appendix 9) The gatekeeper unit 8. The language model utilization device according to claim 1, wherein an instruction statement indicating a method of information compression is added to the first prompt, and the second output information obtained by compressing the first prompt in accordance with the instruction statement is acquired as the second prompt. (Appendix 10) the language model utilization device utilizes a plurality of types of second language models as the second language model, The gatekeeper unit 10. The language model utilization device according to claim 1, wherein the first prompt is input to each of the plurality of types of second language models, a plurality of pieces of second output information are obtained from each of the plurality of types of second language models, and one of the second prompts is obtained from the plurality of pieces of second output information by majority vote. (Appendix 11) 11. The language model utilization device according to claim 1, wherein each of the first language model and the second language model is a generative AI (Artificial Intelligence). (Appendix 12) the first prompt includes a question sentence and has reference information to be used in answering the question sentence added thereto; The gatekeeper unit acquiring the first prompt, inputting the first prompt into the second language model, and acquiring second output information that is information output from the second language model as a second prompt including the question sentence and to which compressed information obtained by compressing the reference information is added; The answer generation unit 3. The language model utilization device according to claim 1, wherein the second prompt is input to the first language model, and first output information output from the first language model is obtained as an answer to the first prompt. (Appendix 13) 1. A language model utilization method used in a language model utilization device that utilizes a first language model and a second language model that is smaller in scale than the first language model, a computer acquires a first prompt, inputs the first prompt into the second language model, and acquires second output information, which is information output from the second language model, as a second prompt obtained by compressing the first prompt; A language model utilization method in which a computer inputs the second prompt into the first language model and obtains first output information output from the first language model as a response to the first prompt. (Appendix 14) A language model utilization program used in a language model utilization device that utilizes a first language model and a second language model that is smaller in scale than the first language model, a gatekeeper process that acquires a first prompt, inputs the first prompt into the second language model, and acquires second output information that is information output from the second language model as a second prompt obtained by compressing the first prompt; an answer generation process of inputting the second prompt into the first language model and acquiring first output information output from the first language model as an answer to the first prompt; A language model utilization program that causes a computer to execute the above. [Explanation of symbols]

[0076] 51 first prompt, 512 second output information, 52 second prompt, 521 first output information, 53 answer, 61 first language model, 62 second language model, 71 question sentence, 72 reference information, 73 compressed information, 100 language model utilization device, 110 gatekeeper unit, 120 answer generation unit, 150 memory unit, 200 artificial intelligence, 500 language model utilization system, 909 electronic circuit, 910 processor, 921 memory, 922 auxiliary storage device, 930 input interface, 940 output interface, 950 communication device.

Claims

1. 1. A language model utilization device that utilizes a first language model and a second language model that is smaller in scale than the first language model, a gatekeeper unit that acquires a first prompt, inputs the first prompt into the second language model, and acquires second output information, which is information output from the second language model, as a second prompt obtained by compressing the first prompt; an answer generation unit that inputs the second prompt to the first language model and acquires first output information output from the first language model as an answer to the first prompt; Equipped with the second language model includes dummy knowledge for checking whether prompt injection is performed; The gatekeeper unit A language model utilization device that determines whether or not the second output information includes the dummy knowledge, and discards the second output information if the dummy knowledge is included.

2. A language model utilization device that utilizes a first language model and a second language model that is smaller in scale than the first language model, a gatekeeper unit that acquires a first prompt, inputs the first prompt into the second language model, and acquires second output information, which is information output from the second language model, as a second prompt obtained by compressing the first prompt; an answer generation unit that inputs the second prompt to the first language model and acquires first output information output from the first language model as an answer to the first prompt; Equipped with the language model utilization device utilizes a plurality of types of second language models as the second language model, The gatekeeper unit a language model utilization device that inputs the first prompt to each of the plurality of types of second language models, obtains a plurality of pieces of second output information output from each of the plurality of types of second language models, and obtains one of the second prompts from the plurality of pieces of second output information by majority vote.

3. The gatekeeper unit 3. The language model utilization device according to claim 1, wherein the second output information obtained by compressing the first prompt by reducing the number of tokens of the first prompt is acquired as the second prompt.

4. The gatekeeper unit 3. The language model utilization device according to claim 1, wherein the second output information is obtained as the second prompt by translating the language of the first prompt into a language different from the language of the first prompt to reduce the number of tokens of the first prompt.

5. A language model utilization device that utilizes a first language model and a second language model that is smaller in scale than the first language model, a gatekeeper unit that acquires a first prompt, inputs the first prompt into the second language model, and acquires second output information, which is information output from the second language model, as a second prompt obtained by compressing the first prompt; an answer generation unit that inputs the second prompt to the first language model and acquires first output information output from the first language model as an answer to the first prompt; Equipped with The gatekeeper unit a language model utilization device that adds to the first prompt an instruction sentence that instructs a search for information to be used in answering the first prompt from the first prompt, extracts information to be used in answering the first prompt from the first prompt, and converts a format of the information to be used in answering the first prompt, thereby acquiring, as the second prompt, the second output information that has been compressed.

6. The gatekeeper unit 10. The language model utilization device according to claim 1, wherein an external prompt check function is used to determine whether prompt injection has been performed in the first prompt, and when it is determined that prompt injection has been performed, the first prompt is discarded.

7. the first prompt includes a method of information compression; The gatekeeper unit 3. The language model utilization device according to claim 1, wherein the second output information obtained by compressing the first prompt using the information compression method included in the first prompt is acquired as the second prompt.

8. The gatekeeper unit 6. The language model utilization device according to claim 1, wherein an instruction statement indicating a method of information compression is added to the first prompt, and the second output information obtained by compressing the first prompt in accordance with the instruction statement is acquired as the second prompt.

9. 6. The language model utilization device according to claim 1, wherein each of the first language model and the second language model is a generative AI (Artificial Intelligence).

10. the first prompt includes a question sentence and has reference information to be used in answering the question sentence added thereto; The gatekeeper unit acquiring the first prompt, inputting the first prompt into the second language model, and acquiring second output information that is information output from the second language model as a second prompt including the question sentence and to which compressed information obtained by compressing the reference information is added; The answer generation unit 6. The language model utilization device according to claim 1, wherein the second prompt is input to the first language model, and first output information output from the first language model is obtained as a response to the first prompt.

11. 1. A language model utilization method used in a language model utilization device that utilizes a first language model and a second language model that is smaller in scale than the first language model, a computer acquires a first prompt, inputs the first prompt into the second language model, and acquires second output information, which is information output from the second language model, as a second prompt obtained by compressing the first prompt; a computer inputting the second prompt into the first language model and obtaining first output information output from the first language model as a response to the first prompt; the second language model includes dummy knowledge for checking whether prompt injection is performed; The computer determines whether the second output information includes the dummy knowledge, and discards the second output information if the dummy knowledge is included.

12. 1. A language model utilization program used in a language model utilization device that utilizes a first language model and a second language model that is smaller in scale than the first language model, a gatekeeper process that acquires a first prompt, inputs the first prompt into the second language model, and acquires second output information, which is information output from the second language model, as a second prompt obtained by compressing the first prompt; an answer generation process of inputting the second prompt into the first language model and acquiring first output information output from the first language model as an answer to the first prompt; A language model utilization program that causes a computer to execute the second language model includes dummy knowledge for checking whether prompt injection is performed; The gatekeeper process includes: a language model utilization program that determines whether the second output information includes the dummy knowledge, and discards the second output information if the dummy knowledge is included;

13. 1. A language model utilization method used in a language model utilization device that utilizes a first language model and a second language model that is smaller in scale than the first language model, a computer acquires a first prompt, inputs the first prompt into the second language model, and acquires second output information, which is information output from the second language model, as a second prompt obtained by compressing the first prompt; a computer inputting the second prompt into the first language model and obtaining first output information output from the first language model as a response to the first prompt; the language model utilization device utilizes a plurality of types of second language models as the second language model, The computer A language model utilization method comprising: inputting the first prompt into each of the plurality of types of second language models; obtaining a plurality of pieces of second output information output from each of the plurality of types of second language models; and obtaining one of the second prompts from the plurality of pieces of second output information by majority vote.

14. 1. A language model utilization program used in a language model utilization device that utilizes a first language model and a second language model that is smaller in scale than the first language model, a gatekeeper process that acquires a first prompt, inputs the first prompt into the second language model, and acquires second output information, which is information output from the second language model, as a second prompt obtained by compressing the first prompt; an answer generation process of inputting the second prompt into the first language model and acquiring first output information output from the first language model as an answer to the first prompt; A language model utilization program that causes a computer to execute the following: the language model utilization device utilizes a plurality of types of second language models as the second language model, The gatekeeper process includes: a language model utilization program that inputs the first prompt to each of the plurality of types of second language models, obtains a plurality of pieces of second output information output from each of the plurality of types of second language models, and obtains one of the second prompts from the plurality of pieces of second output information by majority vote.

15. 1. A language model utilization method used in a language model utilization device that utilizes a first language model and a second language model that is smaller in scale than the first language model, a computer acquires a first prompt, inputs the first prompt into the second language model, and acquires second output information, which is information output from the second language model, as a second prompt obtained by compressing the first prompt; a computer inputting the second prompt into the first language model and obtaining first output information output from the first language model as a response to the first prompt; The computer A method of utilizing a language model, comprising: adding to the first prompt an instruction statement that instructs a search for information to be used in answering the first prompt from the first prompt; extracting information to be used in answering the first prompt from the first prompt; and converting the format of the information to be used in answering the first prompt, thereby obtaining, as the second prompt, the second output information that has been compressed.

16. 1. A language model utilization program used in a language model utilization device that utilizes a first language model and a second language model that is smaller in scale than the first language model, a gatekeeper process that acquires a first prompt, inputs the first prompt into the second language model, and acquires second output information, which is information output from the second language model, as a second prompt obtained by compressing the first prompt; an answer generation process of inputting the second prompt into the first language model and acquiring first output information output from the first language model as an answer to the first prompt; A language model utilization program that causes a computer to execute the following: The gatekeeper process includes: a program for utilizing a language model, the program adding to the first prompt an instruction statement that instructs a search for information to be used in answering the first prompt from the first prompt; extracting information to be used in answering the first prompt from the first prompt; and converting a format of the information to be used in answering the first prompt to obtain, as the second prompt, the second output information that has been compressed.

Images