Information processing device and information processing method

Abstract

An information processing device according to one embodiment of the present invention comprises: an acquisition unit that acquires a current context comprising an input; a log analysis unit that extracts a plurality of examples that correspond to the current context and each comprise an input and an output; a similarity degree calculation unit that calculates the similarity degree between the current context and the extracted examples; a minority example selection unit that adopts an example having the highest similarity degree as a minority example, calculates the similarity degree between the minority example and examples not adopted as the minority example, and adopts only a prescribed number of examples having the low similarity degrees as minority examples; a prompt creation unit that creates, on the basis of the current context and the minority examples, a minority example prompt that can be inputted to a large-scale language model; and an output control unit that outputs the minority example prompt.

Description

Information processing device and information processing method 【0001】 This invention relates to an information processing device and an information processing method. 【0002】 In the operation of ICT (Information and Communication Technology) systems, operator assistance technologies using Large Language Models (LLMs) are expected to become widespread. For example, in cloud or network fault diagnosis, it is being considered to have LLMs generate explanations of root causes from fault information. 【0003】 To suppress hallucination in LLM and improve the accuracy of root cause inference, similar cases to the current failure information are extracted from the history of stored past failure information and root causes, and these are included as specific examples in the prompt. This method of providing not only instructions to LLM but also specific output examples for those instructions is called a small-example prompt, and it is widely used as a technique to improve the inference accuracy of LLM. 【0004】 For example, Non-Patent Document 1 discloses a technique for sampling examples that maximize the diversity of examples in the LLM's small example prompt. By using a variety of examples, more useful information can be provided to the LLM. 【0005】 Non-patent document 2 discloses a technique for cloud-based fault diagnosis that searches a database containing past fault information and root causes for examples similar to the fault information in a query. It also discloses a technique that uses text embedding to obtain several examples that are semantically closest and use them as examples for prompts. 【0006】 Non-patent document 3 discloses a technique for searching the history of past agents that are most similar to the current context in the prompt of an LLM agent. It also discloses a technique for obtaining the most semantically similar example using text embedding and using it as an example in the prompt. 【0007】Zhao Yang, Yuanzhe Zhang, Dianbo Sui, Cao Liu, Jun Zhao, and Kang Liu. 2023. Representative Demonstration Selection for In-Context Learning with Two-Stage Determinantal Point Process. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5443-5456, Singapore. Association for Computational Linguistics.Dylan Zhang, Xuchao Zhang, Chetan Bansal, Pedro Las-Casas, Rodrigo Fonseca, and Saravan Rajmohan. 2024. LM-PACE: Confidence Estimation by Large Language Models for Effective Root Causing of Cloud Incidents. In Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering (FSE 2024). Association for Computing Machinery, New York, NY, USA, 388-398. https: / / doi.org / 10.1145 / 3663529.3663858Kagaya, Tomoyuki, Thong Jing Yuan, Yuxuan Lou, Jayashree Karlekar, Sugiri Pranata, Akira Kinose, Koki Oguri, Felix Wick and Yang You. “RAP: Retrieval-Augmented Planning with Contextual Memory for Multimodal LLM Agents.” ArXiv abs / 2402.03610 (2024): n. pag. 【0008】 Non-patent document 1 assumes that it will cover all possible examples to solve all problems. However, in actual problems in the operation of ICT systems, the problem space is so vast that it is difficult to include all examples in the prompts. 【0009】 Furthermore, Non-Patent Document 2 or Non-Patent Document 3 extracts the example most similar to the current context from a broad range of issues. However, this does not take into account the diversity of examples, and if the diversity of the example output from a small number of example prompts is low, there is a risk of leading to simplistic inferences. 【0010】 This invention was made in view of the above circumstances, and its purpose is to provide a technique that can extract examples relevant to the current context so as to fit within the prompt size, and that can suppress erroneous reasoning. 【0011】 To solve the above problems, an information processing device according to one aspect of the present invention comprises: an acquisition unit that acquires the current context having an input; a log analysis unit that extracts a plurality of examples corresponding to the current context and having inputs and outputs; a similarity calculation unit that calculates the similarity between the current context and the extracted examples; a small number of example selection unit that selects the example with the highest similarity as a small number of examples, calculates the similarity between the small number of examples and the examples not selected as small number of examples, and selects a predetermined number of examples with low similarity as small number of examples; a prompt creation unit that creates a small number of example prompt that can be input to a large-scale language model based on the current context and the small number of examples; and an output control unit that outputs the small number of example prompt. 【0012】 According to one aspect of this invention, it is possible to extract examples that are relevant to the current context so as to fit within the prompt size, and to suppress erroneous reasoning. 【0013】Figure 1 is a diagram showing examples of output when using a highly diverse example and a low-diversity example in the above settings. Figure 2 is a block diagram showing an example of the hardware configuration of the information processing device according to the embodiment. Figure 3 is a block diagram showing the software configuration of the information processing device according to the embodiment in relation to the hardware configuration shown in Figure 2. Figure 4 is a flowchart showing an example of the operation of the information processing device that generates a small number of example prompts with increased output diversity in the example according to the embodiment. Figure 5 is a conceptual diagram showing an example of applying the first specific example according to the embodiment to the log analysis unit. Figure 6 is a conceptual diagram showing an example of selecting a small number of examples from the first specific example according to the embodiment. Figure 7 is a conceptual diagram showing an example of applying the second specific example according to the embodiment to the log analysis unit. Figure 8 is a conceptual diagram showing an example of selecting a small number of examples from the second specific example according to the embodiment. 【0014】 Next, embodiments of the present invention will be described below with reference to the drawings. In the following embodiments, parts with the same number will be assumed to perform the same function, and therefore repeated explanations will be omitted. For example, when there are multiple identical or similar elements, a common reference numeral may be used to describe each element without distinction, or a sub-number may be used in addition to the common reference numeral to describe each element separately. 【0015】 (Small Example Prompt) First, let's explain the overview of the small example prompt. In the small example prompt, in a problem where you infer y from x, the input and output examples { (x １ , y １ ) ... (x Ｎ , y Ｎ )} is given. Here, N is any positive integer. From this, select a decimal example prompt and give it to LLMπ. LLM is a decimal example {(x １ , y １ ) ... (x ｌ , y ｌ Refer to} and infer y from x. Here, l is an integer less than or equal to N. 【0016】The size that can be described in the prompt is limited, and it is impossible to describe all the examples stored in a database or the like in the prompt. Therefore, it is necessary to select a small number of example prompts useful for inference. Here, the above description can be expressed by the following formula. 【0017】 π(y|x １ , y １ 、... x ｌ , y ｌ , x) Here, as described above, y is the output, and x １ , y １ 、... x ｌ , y ｌ are a small number of example prompts, and x indicates the input of the problem. 【0018】 (Problems when the diversity of the outputs of the example prompts is low) Next, the problems when the diversity of the outputs of the example prompts is low will be described. First, assume that the problems can be classified into several types. The input and output texts of the example problems have parts that are common (common parts) and parts that are not common (non-common parts) to the problem types. 【0019】 FIG. 1 is a diagram showing examples of outputs when using examples with high diversity and examples with low diversity in the above settings. As shown in FIG. 1, consider the case where, as a result of selecting the two examples closest to the current context [common part B], the outputs of those examples match. At this time, as shown in the left diagram of FIG. 1, the LLM should generate an output including [non-common part c] in accordance with the [non-common part C] of the input. However, it is difficult to interpret which part of the input corresponds to the non-common part of the output of the example. Therefore, if the LLM interprets that it should output [non-common part a] from [common part B], it will lead to an incorrect output as shown in the right diagram of FIG. 1. 【0020】 Therefore, in the present embodiment, by increasing the diversity of the outputs of the examples, the correspondence between the input and output is clarified, and incorrect inferences are suppressed. 【0021】(Configuration) Next, the hardware and software configuration of the information processing device according to the embodiment will be described. Figure 2 is a block diagram showing an example of the hardware configuration of the information processing device 100 according to the embodiment. The information processing device 100 is a computer that analyzes input data, generates output data, and outputs it. For example, the information processing device 100 is installed in any location set by the user who manages the information processing device 100. 【0022】 As shown in Figure 2, the information processing device 100 comprises a control unit 1, a program storage unit 2, a data storage unit 3, a communication interface 4, and an input / output interface 5. The control unit 1, program storage unit 2, data storage unit 3, communication interface 4, and input / output interface 5 are connected to each other via a bus so as to be able to communicate with each other. Furthermore, the communication interface 4 may be connected to an external device so as to be able to communicate with it via a network. In addition, the input / output interface 5 is connected to an input device 51 and an output device 52 so as to be able to communicate with it. 【0023】 The control unit 1 controls the information processing device 100. The control unit 1 includes a hardware processor such as a central processing unit (CPU). For example, the control unit 1 may be an integrated circuit capable of executing various programs. 【0024】 The program storage unit 2 can use a combination of non-volatile memory that allows writing and reading at any time, such as EPROM (Erasable Programmable Read Only Memory), HDD (Hard Disk Drive), and SSD (Solid State Drive), as a storage medium, and non-volatile memory such as ROM (Read Only Memory). The program storage unit 2 stores programs necessary to execute various processes. In other words, the control unit 1 can realize various controls and operations by reading and executing programs stored in the program storage unit 2. 【0025】The data storage unit 3 is a storage device that uses a combination of non-volatile memory, such as an HDD or memory card, which allows for writing and reading at any time, and volatile memory, such as RAM (Random Access Memory), as storage media. The data storage unit 3 is used to store data acquired and generated during the process in which the control unit 1 executes a program and performs various processing. 【0026】 The communication interface 4 includes one or more wired or wireless communication modules. For example, the communication interface 4 includes a communication module that connects to an external device via a network, either wired or wirelessly. The communication interface 4 may also include a wireless communication module that connects to an external device wirelessly, such as a Wi-Fi access point and a base station. Furthermore, the communication interface 4 may include a wireless communication module that connects to an external device wirelessly using short-range wireless technology. In other words, the communication interface 4 can be any general communication interface that can communicate with an external device and send and receive various types of information under the control of the control unit 1. 【0027】 The input / output interface 5 is connected to the input device 51 and the output device 52, etc. The input / output interface 5 is an interface that enables the transmission and reception of information between the input device 51 and the output device 52. The input / output interface 5 may be integrated with the communication interface 4. For example, the information processing device 100 and at least one of the input device 51 and the output device 52 may be wirelessly connected using short-range wireless technology, and information may be transmitted and received using said short-range wireless technology. 【0028】 The input device 51 may include, for example, a keyboard or pointing device for the user to input various information to the information processing device 100. The input device 51 may also include a reader for reading data to be stored in the program storage unit 2 or the data storage unit 3 from a memory medium such as a USB memory, or a disk device for reading such data from a disk medium. 【0029】The output device 52 includes a display that shows the results calculated by the control unit 1, for example, a small number of example prompts. The output device 52 also includes a printer that prints the information displayed on the display. 【0030】 Figure 3 is a block diagram showing the software configuration of the information processing device 100 according to the embodiment, in relation to the hardware configuration shown in Figure 2. As shown in Figure 3, the control unit 1 includes an input data acquisition unit 11, a log analysis unit 12, a similarity calculation unit 13, a small number of example selection unit 14, a prompt creation unit 15, and an output control unit 16. 【0031】 The input data acquisition unit 11 is an acquisition unit that acquires the current context. The current context is composed of inputs such as problem or failure information, and the LLM is used to derive an output for said input. For example, the output contains the root cause of the problem or failure information. The input data acquisition unit 11 acquires the current context from the input device 51 via the input / output interface 5 or from an external device connected via the network through the communication interface 4. The input data acquisition unit 11 stores the acquired current context in the input data storage unit 31. 【0032】 The log analysis unit 12 is an analysis unit that extracts examples corresponding to the current context from the example database 32 (described later) and analyzes those examples. For example, an example is a past case of a problem corresponding to the current context and includes inputs and outputs. The log analysis unit 12 identifies templates and parameters corresponding to the current context and each example. For example, the log analysis unit 12 may identify the templates and parameters of the input and output of an example. 【0033】 The similarity calculation unit 13 is a calculation unit that calculates the similarity between the current context and an example. For example, the similarity calculation unit 13 searches for the example template that is most similar to the template of the current context. For example, the similarity calculation unit 13 calculates the similarity between the sentences of the template of the current context and the template of the example. Details of the similarity calculation method will be described later. 【0034】The small number of example selection unit 14 is a selection unit that selects a small number of examples from the examples to be used to create a prompt. For example, the small number of example selection unit 14 selects the example with the highest similarity calculated by the similarity calculation unit 13 as the small number of examples. Furthermore, the small number of example selection unit 14 calculates the similarity between the output template of the small number of examples and the output template of the examples, and selects a predetermined number of examples with low similarity as the small number of examples. By selecting the small number of examples and examples with low similarity in this way, the diversity of the output can be increased. 【0035】 The prompt creation unit 15 is a unit that creates a few-example prompt. For example, the prompt creation unit 15 creates a few-example prompt based on the current context and the few-example selection unit 14. 【0036】 The output control unit 16 is a control unit that controls the display of the few example prompts created by the prompt creation unit 15 on the display of the output device 52. The output device 52 may also be controlled to transmit the few example prompts to an external device that performs LLM via the communication interface 4 and the network. 【0037】 The data storage unit 3 comprises an input data storage unit 31 and an example database 32. The input data storage unit 31 is a storage unit used to store the current context, etc., acquired by the input data acquisition unit 11. The example database 32 stores past cases corresponding to the current context (for example, past problems or failure information that serve as input, and the root causes that serve as output for these). In other words, the example database 32 stores past examples. 【0038】 (Operation) Next, the operation of the information processing device 100, which generates a small number of example prompts with increased diversity in the output of the examples used to create the prompts, will be described. Figure 4 is a flowchart showing an example of the operation of the information processing device 100, which generates a small number of example prompts with increased diversity in the output of the examples according to the embodiment. The operation of this flowchart is realized when the control unit 1 of the information processing device 100 reads and executes the program stored in the program storage unit 2. 【0039】The operations shown in this flowchart are initiated when the current context x, which is an instruction input to the LLM, is input or received by the user managing the information processing device 100 from the input device 51 via the input / output interface 5, or from an external device via the communication interface 4. Here, the current context x can also be classified into several problem types. 【0040】 In step ST101, the input data acquisition unit 11 acquires the current context x. The input data acquisition unit 11 acquires the current context x to be input to the LLM from the input device 51 via the input / output interface 5 or from an external device connected via the network through the communication interface 4. The input data acquisition unit 11 stores the current context x in the input data storage unit 31. 【0041】 In step ST102, the log analysis unit 12 identifies the template and parameters for the current context x. The log analysis unit 12 identifies the template T and parameters P for the current context x (x Ｔ , x Ｐ The log analysis unit 12 identifies the current context x, in which the template and parameters have been identified, and outputs it to the similarity calculation unit 13. 【0042】 In step ST103, the log analysis unit 12 extracts examples. For example, the log analysis unit 12 extracts all examples in advance in order to extract a template in step ST104. The log analysis unit 12 extracts N examples, which are positive integers, as all examples. 【0043】 In step ST104, the log analysis unit 12 identifies the example template and parameters. The log analysis unit 12 identifies the example {x ｎ , y ｎ} Ｎ ｎ＝１ Each of the common parts that are common to the problem type and the non-common parts that are not common to the problem type are identified as templates and parameters, respectively. For example, the log analysis unit 12 identifies the input and output templates T and parameters P of the nth example as (x Ｔ ｎ , x Ｐ ｎ , y Ｔｎ , y Ｐ ｎ The log analysis unit 12 identifies the template and parameters as follows. The log analysis unit 12 outputs examples of identified templates and parameters to the similarity calculation unit 13. 【0044】 Note that the processing in step ST102 and the series of processing steps ST103 to ST104 may be performed in any order, or they may be processed in parallel. 【0045】 In step ST105, the similarity calculation unit 13 searches for examples similar to the template of the current context x. The similarity calculation unit 13 searches for input templates x of the current context x. Ｔ Input template x that is most similar to the example Ｔ ｍ The search function is used to find the input template x of the current context x. Here, m is any positive integer less than or equal to N. For example, the similarity calculation unit 13 searches for the input template x of the current context x. Ｔ And example {x ｎ , y ｎ} Ｎ ｎ＝１ Each template x Ｔ ｎ The similarity calculation unit 13 calculates the similarity of the sentences. The similarity calculation unit 13 may use semantic search using sentence embeddings as a method for calculating similarity. In this case, the similarity calculation unit 13 converts the sentences into sentence embeddings and calculates their similarity. For example, the similarity calculation unit 13 uses the input template x of the current context x. Ｔ And example {x ｎ , y ｎ} Ｎ ｎ＝１ Input template x Ｔ ｎ The cosine similarity is calculated using the embedding vectors of each sentence. Note that the method for calculating similarity is not limited to the above example; any method capable of calculating sentence similarity may be used. As described above, searching for examples using a template enables robust searching unaffected by parameters. The similarity calculation unit 13 outputs the current context, examples, and similarity to the small number of example selection unit 14. 【0046】 In step ST106, the small number of example selection unit 14 selects the most similar example (xｍ , y ｍ Select ) and add it to the few examples. The few example selection unit 14 selects example {x ｎ , y ｎ} Ｎ ｎ＝１ Of the similarity scores calculated for each, the example with the highest similarity (x ｍ , y ｍ ) are selected as a small number of examples. Here, the small number of example selection unit 14 may select the top k examples as a small number of examples using a predetermined value k. The similarity calculation unit 13 calculates example {x ｎ , y ｎ} Ｎ ｎ＝１ and a few selected examples (x ｍ , y ｍ The results are output to the small number of example selection unit 14. 【0047】 In step ST107, the small number of example selection unit 14 selects the selected example (x ｍ , y ｍ ) Output template y Ｔ ｍ Similar example {x ｎ , y ｎ} Ｎ ｎ＝１ Output template y Ｔ ｌ Select multiple items. The small number of example selection unit 14 selects the examples (x) using the same or different method as calculated in step ST105. ｍ , y ｍ ) Output template y Ｔ ｍ And example {x ｎ , y ｎ} Ｎ ｎ＝１ Each output template y Ｔ ｌ The similarity is calculated. Here, l is any positive integer less than or equal to N. If there are multiple minority examples, the minority example selection unit 14 selects an example that is similar to each of the multiple minority examples. 【0048】In step ST108, the minority example selection unit 14 adopts, as minority examples, examples of output templates with low similarity to the output template of the minority examples. In steps ST107 to ST108, the minority example selection unit 14 adopts, as minority examples, examples where the templates match but the parameters deviate. In step ST106, the similarity calculation unit 13 selects the example (x ｍ ,y ｍ ) that is most similar in terms of the template of the current context (situation). At this point, one minority example to be adopted is determined. However, the example (x ｍ ,y ｍ ) that is a minority example includes parameters. In the command input to the LLM, it is desirable to appropriately modify the parameters according to the situation while following the example (x ｍ ,y ｍ ). Therefore, the minority example selection unit 14 adopts, as minority examples, examples that give diversity to the parameters. As described above, the template is determined by the example (x ｍ ,y ｍ ) adopted by the similarity calculation unit 13. Therefore, the minority example selection unit 14 selects the example with the most dissimilar content (lowest similarity) from the examples having the same template. That is, the minority example selection unit 14 selects examples where the templates are similar and the parameters are not similar for the second and subsequent examples. 【0049】 The minority example selection unit 14 refers to the similarity calculated in step ST107 and selects, as minority examples, the output template y Ｔ ｍ of the example with the lowest similarity to the output template of the minority examples. Also, depending on the template, the parameter y ｌ ｐ ​Since the number of elements differs, it is difficult to calculate the similarity of the parameters. Therefore, the small number of example selection unit 14 calculates the similarity of the entire surface layer y of the examples. In this case, since all the examples being compared have matching (or similar) templates, the example with the lowest overall surface layer similarity will also have a low parameter similarity. In this way, by adopting examples with low similarity between the original small number of examples and the output template as new small number of examples, the diversity of the output parameters can be increased. If multiple examples are selected as small number of examples in step ST106, the small number of example selection unit 14 adopts the example with the lowest similarity for each of the selected small number of examples. 【0050】 In step ST109, the small number of cases selection unit 14 determines whether the number of small numbers is greater than or equal to a certain number. If it determines that the number of small numbers is greater than or equal to a certain number, the process proceeds to step ST110. On the other hand, if it determines that the number of small numbers is not greater than or equal to a certain number, the process returns to step ST108. 【0051】 In step ST110, the prompt creation unit 15 creates a prompt based on a small number of examples. The prompt creation unit 15 creates a small number of example prompt that can be input to the LLM based on the current context and the small number of examples. The prompt creation unit 15 outputs the created small number of example prompt to the output control unit 16. The output control unit 16 controls the display of the output device 52 to display the small number of example prompt via the input / output interface 5. The output control unit 16 may also control the transmission of the small number of example prompt to an external device running the LLM via the communication interface 4. Furthermore, if the LLM is running on the information processing device 100, the output control unit 16 may also control the input of the small number of example prompt to the LLM. 【0052】 (First Specific Example) Next, we will explain the first specific example of adopting the above process. In the first specific example, we will explain an example of creating a prompt to generate an appropriate countermeasure from the given fault information (current context) by referring to the history of fault information and countermeasures. 【0053】Figure 5 is a conceptual diagram showing an example of applying the first specific example according to the embodiment to the log analysis unit 12. The left side of Figure 5 represents a past example, and the right side represents the current context. 【0054】 As shown in Figure 5, the current context x and past cases extracted by the log analysis unit 12 are input to the log analysis unit 12. In the example in Figure 5, the current context is "Input: An HTTP error occurred when connecting to the service catalogogue in the service frontend. The host is accessible, but port 80 is not registered in the firewall's allow list." 【0055】 The example is: "Input: An HTTP error occurs when connecting the adservice service to the recommend service. The host address is accessible, but port 8080 used for communication is not registered in the firewall's allow list. Output: The adservice service uses the ufw command to register port 8080 in the allow list." 【0056】 As shown in Figure 5, the log analysis unit 12 extracts the template for the current context x as "Input template: An HTTP error occurred when connecting to * in *. * * is not registered in the firewall's allow list." Here, * represents a parameter. 【0057】 Similarly, the log analysis unit 12 extracts the example template as "Input template: An HTTP error occurred when connecting to * at *. * * is not registered in the firewall's allow list. Output template: Use the ufw command at * to register * in the allow list." 【0058】 Then, as described in step ST105 with reference to Figure 4, the similarity calculation unit 13 calculates the template x of the current context x. Ｔ The template most similar to x Ｔ ｍThe search is performed. As described above, the similarity calculation unit 13 calculates the cosine similarity of the embedding vectors of the text in the input template and selects a small number of examples with the highest cosine similarity. 【0059】 Figure 6 is a conceptual diagram showing a selection of a small number of specific examples according to the embodiment. The left side of Figure 6 shows the examples adopted for the small number of examples, and the right side shows the candidates for the small number of examples. 【0060】 As shown in Figure 6, the example adopted as a minority example is: "Input: An HTTP error occurred when connecting to the service recommend service in the service adservice. The host address is accessible, but port 8080 used for communication is not registered in the firewall's allow list. Output: The ufw command is used in the service adservice to register port 8080 in the allow list." As explained with reference to Figure 4, the minority example selection unit 14 selects several examples that are similar to the output template of the example adopted as a minority example, and adopts the example with the lowest similarity to the output template of the minority example. For example, the right side of Figure 6 shows an example selected by the minority example selection unit 14. 【0061】 The example in the middle right of Figure 6, "Input: An HTTP error occurred when connecting to the service 'catalogue' in the service 'adservice'. Port 8080, used for communication, is registered in the firewall's allow list. Output: The ufw command was used in the service 'adservice' to register port 8080 in the allow list," has a high degree of similarity to the output template of the few examples. Therefore, this example will not be adopted. 【0062】On the other hand, the example in the lower right of Figure 6, "Input: An HTTP error occurred when connecting to the service frontend in the service adservice. Port 80, used for communication, is registered in the firewall's allow list. Output: The ufw command was used in the service adservice to register port 80 in the allow list," has a low degree of similarity to the output template of the minority example. Therefore, this example will be adopted as a minority example. Then, the prompt creation unit 15 creates a minority example prompt based on the current context and the minority example and the example that was designated and adopted. 【0063】 (Second Specific Example) Next, we will explain a second specific example of adopting the above process. In the second specific example, we will explain an example of creating a prompt in the LLM agent prompt to generate the current action by referring to past trajectories. 【0064】 Figure 7 is a conceptual diagram showing an example of applying a second specific example according to the embodiment to the log analysis unit 12. The left side of Figure 7 represents a past example, and the right side represents the current context. 【0065】 As shown in Figure 7, the current context x and past examples extracted by the log analysis unit 12 are input to the log analysis unit 12. In the example in Figure 7, the current context is "Input: Observation: You are in front of bed 1. You see a book 3 on bed 1." 【0066】 The example is: "Input: Observation: You are in front of countertable 2. You see an apple 1 on countertable 2. Output: Action: pick up apple 1 from countertable 2." 【0067】 As shown in Figure 7, the log analysis unit 12 extracts the template for the current context x as "Input: Observation: You are in front of *. You see a * on *." 【0068】Similarly, the log analysis unit 12 extracts the example template as "Input template: Observation: You are in front of *. You see an * on *. Output template: Action: pick up * from *." 【0069】 Then, like the process of step ST105 described with reference to FIG. 4, the similarity calculation unit 13 searches for the template x Ｔ that is most similar to the template x Ｔ ｍ of the current context x. As described above, the similarity calculation unit 13 calculates the cosine similarity of the embedding vectors of the sentences in the input template and selects the example with the highest cosine similarity as the minority example. In the second specific example, at this point, the command is determined. 【0070】 FIG. 8 is a conceptual diagram showing a selection example of a minority example in the second specific example according to the embodiment. The left side of FIG. 8 represents the example adopted as the minority example, and the right side represents the candidates for the minority example. 【0071】 As shown in FIG. 8, the example adopted as the minority example is "Input: Observation: You are in front of countertable 2. You see an apple 1 on countertable 2. Output: Action: pick up apple 1 from countertable 2." Like the processes of steps ST107 to ST109 described with reference to FIG. 4, the minority example selection unit 14 selects a plurality of examples similar to the output template of the example adopted as the minority example, and adopts the example with a low similarity of the output template of the minority example among those examples. For example, the right side of FIG. 6 shows the example selected by the minority example selection unit 14. 【0072】The example in the middle right of Figure 8, "Input: Observation: You are in front of countertable 2. You see an apple 1 on countertable 2. Output: Action: pick up apple 1 from countertable 2.", has a high degree of similarity to the few output templates. Therefore, this example will not be adopted. 【0073】 On the other hand, the example in the lower right of Figure 8, "Input: Observation: You are in front of countertable 2. You see a book 1 on countertable 2. Output: Action: pick up book 1 from countertable 2.", has a low degree of similarity to the output template of the few examples. Therefore, this example will be adopted as a few example. Then, the prompt creation unit 15 creates a few example prompt based on the current context and the selected example of the few examples. 【0074】 (Effects) According to the embodiment, the information processing device 100 extracts the example with the highest similarity to the current context, and extracts a predetermined number of examples with low similarity to the extracted example. By doing so, by extracting only a predetermined number of examples, it is possible to extract examples that fit within the prompt size. 【0075】 Furthermore, by selecting examples with low similarity to the most similar example in the current context, the diversity of examples can be increased. This helps to suppress erroneous inferences made by LLM. 【0076】 [Other Embodiments] Furthermore, in the above embodiment, the information processing device 100 can also have the operation of each component constructed as a program, which can then be installed and executed on a computer used as a user device or on a computer used as an external device. 【0077】Furthermore, the method described in the above embodiment can be distributed by storing the program (software means) that can be executed by a computer in a storage medium such as a magnetic disk (floppy disk, hard disk, etc.), an optical disk (CD-ROM, DVD, MO, etc.), or a semiconductor memory (ROM, RAM, flash memory, etc.), and by transmitting it via a communication medium. The program stored on the medium also includes a configuration program that configures the software means (including not only the execution program but also tables and data structures) to be executed by the computer. The computer realizing this device reads the program stored in the storage medium, and, if necessary, constructs the software means using the configuration program, and executes the above-described process by controlling its operation with this software means. The storage medium referred to in this specification is not limited to distribution mediums, but also includes storage mediums such as magnetic disks and semiconductor memories provided inside the computer or in devices connected via a network. 【0078】 In short, this invention is not limited to the embodiments described above, and can be modified in various ways during implementation without departing from its essence. Furthermore, each embodiment may be combined as appropriately as possible, in which case the combined effects can be obtained. Moreover, the embodiments described above include inventions at various stages, and various inventions can be extracted by appropriate combinations of the multiple constituent elements disclosed. 【0079】 100... Information processing device 1... Control unit 11... Input data acquisition unit 12... Log analysis unit 13... Similarity calculation unit 14... Small number of example selection unit 15... Prompt creation unit 16... Output control unit 2... Program storage unit 3... Data storage unit 31... Input data storage unit 32... Example database 4... Communication interface 5... Input / output interface 51... Input device 52... Output device

Claims

1. An information processing device comprising: an acquisition unit that acquires the current context having input; a log analysis unit that extracts a plurality of examples having input and output corresponding to the current context; a similarity calculation unit that calculates the similarity between the current context and the extracted examples; a small number of example selection unit that selects the example with the highest similarity as a small number of examples, calculates the similarity between the small number of examples and the examples not selected as the small number of examples, and selects a predetermined number of examples with low similarity as small number of examples; a prompt creation unit that creates a small number of example prompt that can be input to a large-scale language model based on the current context and the small number of examples; and an output control unit that outputs the small number of example prompt.

2. The log analysis unit identifies templates that are common to the problem types in the current context and the examples, and parameters that are not common to the problem types; and the similarity calculation unit calculates the similarity between the template of the current context and the template of the example, as described in claim 1.

3. The log analysis unit identifies templates that are common to the problem types for the input in the current context and parameters that are not common to the problem types, identifies the templates and parameters for the input and output of the example, and the small number of example selection unit calculates the similarity between the output template of the small number of example and the output template of an example that was not selected for the small number of example, according to claim 1.

4. An information processing method executed by the processor of an information processing device, comprising: acquiring a current context having input; a log analysis unit that extracts a plurality of examples having input and output corresponding to the current context; calculating the similarity between the current context and the extracted examples; adopting the example with the highest similarity as a minority example; calculating the similarity between the minority example and the examples not adopted as minority examples; adopting a predetermined number of examples with low similarity as minority examples; creating a minority example prompt that can be input to a large-scale language model based on the current context and the minority example; and outputting the minority example prompt.

Images