Information processing device, information processing method, and program

Abstract

This information processing device determines, from among a plurality of elements, an element that satisfies a constraint condition and optimizes an objective function, and comprises: a problem setting unit that selects a candidate element from among the plurality of elements on the basis of an output that can be obtained by giving an input from a user to a large language model; and a solving unit that determines, for the candidate element, an element that satisfies the constraint condition and optimizes the objective function.

Description

Information processing device, information processing method, and program

[0001] The present disclosure relates to an information processing device, an information processing method, and a program.

[0002] Combinatorial optimization problems, such as the Traveling Sales Problem (TSP) and the Orienteering Problem (OP), involve finding a solution that maximizes or minimizes an objective function under certain constraints. These problems are NP-hard, and as the number of combinations increases, the computational complexity increases exponentially, making it impossible to find an optimal solution within a realistic timeframe. Therefore, research and development of approximation and heuristic methods to reduce computational time and determine the best possible solution is ongoing.

[0003] Non-Patent Document 1 proposes a recommendation system based on user preferences using a large language model (LLM). LLMs excel at learning a variety of tasks from huge datasets and are expected to accurately capture contextual nuances and user preferences. By updating responses based on prompts from the user, the LLM leads to more appropriate recommendations and improves user satisfaction.

[0004] Jesse Harte, Wouter Zorgdrager, Panos Louridas, Asterios Katsifodimos, Dietmar Jannach, and Marios Fragkoulis. 2023. Leveraging Large Language Models for Sequential Recommendation. In Proceedings of the 17th ACM Conference on Recommender Systems (RecSys '23). Association for Computing Machinery, New York, NY, USA, 1096-1102. https: / / arxiv.org / abs / 2309.09261Zhen Yang, Ming Ding, Qingsong Lv, Zhihuan Jiang, Zehai He, Yuyi Guo, Jinfeng Bai, and Jie Tang. 2023. Gpt can solve mathematical problems without a calculator.arXiv preprint arXiv:2309.03241 (2023). https: / / arxiv.org / abs / 2309.03241

[0005] However, the recommendations or optimizations made by the LLM do not always produce the desired results as intended. Solutions to improve the accuracy of the optimization have been sought, but the results have not been satisfactory.

[0006] In consideration of the above circumstances, an object of the present disclosure is to provide a novel technique for solving combinatorial optimization problems based on user preferences with high accuracy in a short calculation time.

[0007] One aspect of the present disclosure is an information processing device that determines an element from a plurality of elements that satisfies constraint conditions and optimizes an objective function, and that includes: a problem setting unit that selects candidate elements from the plurality of elements based on an output obtained by providing input from a user to a large-scale language model; and a solution-finding unit that determines, for the candidate elements, an element that satisfies the constraint conditions and optimizes the objective function.

[0008] In this way, by narrowing down the elements using a large-scale language model, calculation time can be shortened, and by narrowing down the elements based on the output of a large-scale language model in response to input from the user, it is possible to reflect the user's preferences, and an appropriate solution that suits the user's preferences can be obtained.

[0009] In one embodiment, the problem setting unit may present to the user tentative candidate elements selected from the plurality of elements based on an output obtained by providing a large-scale language model with input from the user, and determine an element selected by the user from the tentative candidate elements as the candidate element. By having the user select the candidate in this manner, the user's preferences can be more reliably reflected, and since the tentative candidate elements are provided by the large-scale language model, the burden on the user can be reduced. Note that in other embodiments, the problem setting unit may determine the elements output by the large-scale language model as candidate elements as they are, or may add elements obtained by other methods to the candidate elements.

[0010] In one embodiment, the problem setter may obtain text describing the tentative candidate elements from the large-scale language model and present the text to the user. For example, when instructing the large-scale language model to return the tentative candidate elements, the problem setter may instruct the large-scale language model to also return text describing the explanation. By presenting the explanation text to the user, the user can determine whether the tentative candidate elements match their preferences, and the user's preferences can be more accurately reflected in the determination of candidate elements.

[0011] In one embodiment, the objective function may be a sum of scores assigned to each of the multiple elements, and the problem setter may determine the score to be assigned to the candidate element based on an output obtained by providing the user's input to the large-scale language model. For example, the problem setter may instruct the large-scale language model to return a degree of match between the user's preferences contained in the user's input and the candidate element, and the degree of match obtained from the large-scale language model may be determined as the score of the candidate element. Note that it is not necessary to determine the entire score based on the output of the large-scale language model; part of the score may be determined based on user input, information from other sources, preset setting information, etc. Formulating a problem based on user preferences is generally time-consuming, but formulating the objective function based on the output of the large-scale language model enables simple processing.

[0012] In one embodiment, the constraints may include a condition that the sum of the costs assigned to each of the plurality of elements is equal to or less than a threshold, and the problem formulation unit may determine the cost to be assigned to the candidate element based on an output obtained by providing the user's input to the large-scale language model. For example, the problem formulation unit may instruct the large-scale language model to return the cost of the candidate element, and the output from the large-scale language model may be determined as the cost. Note that it is not necessary to determine all of the cost based on the output of the large-scale language model; some of the cost may be determined based on user input, information from other sources, preset configuration information, etc. Formulating a problem according to user preferences is generally time-consuming, but formulating constraints based on the output of the large-scale language model enables simple processing.

[0013] The optimization problem to which this aspect can be applied is any combinatorial optimization problem, and examples include travel planning, menu planning, and work shift planning.

[0014] A large language model (LLM) is a machine learning model, also known as generative AI, that has the learning capability to generate versatile content. A large language model refers to a language model that has increased scale in at least one of the computational complexity, data volume, and number of model parameters. In this disclosure, a large language model is defined as a language model with 5 billion or more model parameters. User input may be in any form, such as natural language information or image information, and more specifically, may include information expressed in text, speech, gestures, images, and any combination thereof.

[0015] The present disclosure can be understood as an information processing device or optimization device including at least some of the above means. The present disclosure can also be understood as a method including at least some of the above processing, or a program for realizing such a method, or a recording medium on which such a program is non-temporarily recorded. Furthermore, the present disclosure includes an apparatus or method in which each of the above means and processing is combined with each other to the extent possible.

[0016] According to the present disclosure, it becomes possible to solve combinatorial optimization problems based on user preferences with high accuracy in a short calculation time.

[0017] 1 is a diagram showing an overview of an information processing device according to the present disclosure. A diagram showing a hardware configuration of a control device according to the present disclosure. A flowchart showing an overall flow executed by an information processing device according to the present disclosure. A flowchart showing details of an element narrowing down process according to the present disclosure. A diagram showing examples of a prompt and an LLM answer in the element narrowing down process according to the present disclosure. A flowchart showing details of a score / cost setting process according to the present disclosure. A diagram showing examples of a prompt and an LLM answer in the score / cost setting process according to the present disclosure. A diagram explaining a problem formulation according to the present disclosure. A diagram showing examples of a prompt and an LLM answer in the element narrowing down process according to Example 1 of Modified Example 1 of the present disclosure. A diagram showing examples of a prompt and an LLM answer in the score / cost setting process according to Example 1 of Modified Example 1 of the present disclosure. A diagram showing examples of a prompt and an LLM answer in the element narrowing down process according to Example 2 of Modified Example 1 of Modified Example 1 of the present disclosure. A diagram showing examples of a prompt and an LLM answer in the score / cost setting ... the present disclosure. A diagram showing examples of a prompt and an LLM answer in the element narrowing down process according to Example 3 of Modified Example 1 of the present disclosure. A diagram showing examples of prompts and LLM answers in the element narrowing down process in example 4 of modified example 1 of the present disclosure. A diagram showing examples of prompts and LLM answers in the element narrowing down process in example 4 of modified example 1 of the present disclosure. A diagram showing examples of prompts and LLM answers in the score / cost setting process in example 4 of modified example 1 of the present disclosure. A diagram showing the results of a user questionnaire survey in the evaluation of travel plan creation according to the present disclosure (Experiment 1).

[0018] <Application Example> First, an example of a plan creation device to which the present disclosure is applied will be described. The plan creation device creates a plan that matches the user's preferences by taking the user's preferences into consideration. For example, the plan creation device solves optimization problems such as travel planning, menu planning, and work shift planning, and creates a plan that matches the user's preferences by setting the objective function and constraint conditions according to the user's preferences.

[0019] 1 is a diagram illustrating an overview of a plan creation device 100 according to the present disclosure. The plan creation device 100 has, as functional units, a problem formulation unit 110, a solver 120, and a result output unit 130. The problem formulation unit 110 further has a candidate element selection unit 111, an objective function setting unit 112, and a constraint condition setting unit 113.

[0020] The problem formulation unit 110 has a function of formulating a combinatorial optimization problem, specifically, narrowing down candidate elements and setting objective functions and constraints. The problem formulation unit 110 formulates the problem using a large-scale language model (LLM) 140.

[0021] The candidate element selection unit 111 narrows down the elements using the LLM 140. For example, the candidate element selection unit 111 provides the LLM 140 with a prompt including the user's preferences, receives from the LLM 140 suggestions of elements that match the user's preferences, and determines elements (candidate elements) to be used for the optimization problem based on the suggestions from the LLM 140. For example, the candidate element selection unit 111 may present suggestions from the LLM 140 to the user and select elements selected by the user as candidate elements, or the candidate element selection unit 111 may determine candidate elements from the suggestions from the LLM 140 based on predetermined criteria, or may directly determine the suggestions from the LLM 140 as candidate elements. Note that the candidate element selection unit 111 may query multiple different LLMs 140, or may query the same LLM 140 after assigning different characteristics to it, and may also suggest elements to the user based on multiple responses obtained in this manner.

[0022] The objective function setting unit 112 determines the scores (parameters constituting the objective function) to be assigned to each element using the LLM 140. For example, the objective function setting unit 112 instructs the LLM 140 to return the degree of match between the candidate element and the user's preference, and determines the degree of match obtained from the LLM 140 as the score of the candidate element.

[0023] The constraint condition setting unit 113 determines the cost (parameters constituting the constraint) to be assigned to each element or a combination of elements using the LLM 140. For example, the constraint condition setting unit 113 instructs the LLM 140 to return the cost related to the selection of a candidate element, and determines the cost obtained from the LLM 140 as the cost of the candidate element.

[0024] The candidate element selection unit 111, the objective function setting unit 112, and the constraint condition setting unit 113 enable the determination (narrowing down) of elements that constitute a combinatorial optimization problem and the formulation of the objective function and constraint conditions. After formulating the combinatorial optimization problem, the problem formulation unit 110 provides the formula to the solver 120 to obtain a solution to the problem.

[0025] The solver 120 is a functional unit that solves combinatorial optimization problems and is configured using any existing algorithm. The algorithm adopted by the solver 120 may be determined appropriately depending on the type of optimization problem to be solved, and examples of algorithms that can be adopted include the branch and bound method, dynamic programming, local search, and Edmund's method.

[0026] The result output unit 130 outputs the optimal solution found by the solver 120 in a form such as display on a display device, storage in a storage device, or communication to another device.

[0027] An example of a problem handled by the plan creation device 100 is travel planning. In travel planning, for example, tourist destinations (visiting destinations) and the order in which they are visited are determined so that the total travel and stay time satisfies predetermined constraints and maximizes user satisfaction. In this example, the candidate element selection unit 111 provides the LLM 140 with the user's preferences, such as the user's interests and desired tourist destination requirements, and causes the LLM 140 to suggest candidate tourist destinations that match the user's preferences. The candidate element selection unit 111, for example, presents tourist destinations obtained from the LLM 140 to the user and determines the tourist destination selected by the user as a candidate element. The objective function setting unit 112 also provides the user's preferences to the LLM 140 and causes it to output the user's satisfaction (score) for the candidate elements. The constraint condition setting unit 113 also queries the LLM 140 about the stay time at each tourist destination and determines the stay time as a cost. The travel time between tourist spots is also a cost, but the travel time is acquired from other information sources. The upper limit of the travel time (total of stay time and travel time) may be input by the user, or may be determined by the LLM 140.

[0028] In this way, by narrowing down the candidate elements according to the user's preferences, a solution that matches the user's preferences can be obtained, and the limited number of elements reduces the calculation time. Furthermore, the user's preferences are also reflected in the scores that make up the objective function and the costs that make up the constraints, so a solution that matches the user's preferences can be obtained. Furthermore, communication with the LLM 140 can be done using natural language (text), making it easy and convenient for the user.

[0029] 2 schematically illustrates an example of the hardware configuration of the plan creation device 100 according to this embodiment. As illustrated in Fig. 2, the plan creation device 100 according to this embodiment is a computer (information processing device) in which a control unit 201, a storage unit 202, an input device 205, an output device 206, a communication interface 207, and a drive 208 are electrically connected.

[0030] The control unit 201 includes a hardware processor such as a central processing unit (CPU), a random access memory (RAM), and a read-only memory (ROM), and is configured to execute information processing based on programs and various data. The control unit 201 (CPU) is an example of a processor resource.

[0031] The storage unit 202 is an example of a memory resource, and is configured, for example, by a hard disk drive, a solid state drive, etc. In this embodiment, the storage unit 202 stores various types of information such as a plan creation program 203, prompt information 204, etc.

[0032] The planning program 203 is a program for causing the planning device 100 to execute information processing for a combinatorial optimization problem. The planning program 203 includes a series of instructions for the information processing. The prompt information 204 is information to be included in the instruction statement (prompt) 140 that queries the LLM 140, and corresponds to a system prompt, which will be described later.

[0033] The input device 205 is a device for inputting, such as a mouse, keyboard, keypad, or touch panel device. The output device 206 is a device for outputting, such as a display or speaker. A user can operate the plan creation device 100 by using the input device 205 and the output device 206. The input device 205 and the output device 206 may be integrated into one device, such as a touch panel display.

[0034] The communication interface 207 is, for example, a wired LAN (Local Area Network) module, a wireless LAN module, etc., and is an interface for performing wired or wireless communication via a network. The plan creation device 100 can perform data communication with the LLM 140 and other computers via the communication interface 207.

[0035] The drive 208 is, for example, a CD drive, a DVD drive, or the like, and is a drive device for reading various information such as programs stored in a storage medium 209. At least one of the plan creation program 203 and the prompt information 204 may be stored in the storage medium 209.

[0036] The storage medium 209 is a medium that stores information such as programs electrically, magnetically, optically, mechanically, or chemically so that the various information stored therein can be read by a computer or other device, machine, etc. The plan creation device 100 may acquire at least one of the plan creation program 203 and the prompt information 204 from the storage medium 209.

[0037] 2 illustrates a disk-type storage medium such as a CD or DVD as an example of the storage medium 209. However, the type of the storage medium 209 is not limited to a disk type, and may be a type other than a disk. Examples of storage media other than a disk type include semiconductor memories such as flash memories. The type of the drive 208 may be selected appropriately depending on the type of the storage medium 209.

[0038] Note that, with regard to the specific hardware configuration of the plan creation device 100, components may be omitted, replaced, or added as appropriate depending on the embodiment. For example, the control unit 201 may include multiple hardware processors. The hardware processor may be configured with a microprocessor, a field-programmable gate array (FPGA), a digital signal processor (DSP), or the like. The storage unit 202 may be configured with RAM and ROM included in the control unit 201. At least one of the input device 205, the output device 206, the communication interface 207, and the drive 208 may be omitted. The plan creation device 100 may be configured with multiple computers. In this case, the hardware configurations of the computers may or may not be the same. Furthermore, the plan creation device 100 may be an information processing device designed specifically for the services provided, as well as a general-purpose server device, a general-purpose personal computer (PC), an industrial PC, or the like.

[0039] 1 , the plan creation device 100 functions as a problem formulation unit 110, a solver (solution-finding unit) 120, and a result output unit 130. The problem formulation unit 110 further includes a candidate element selection unit 111, an objective function setting unit 112, and a constraint condition setting unit 113.

[0040] The problem formulation unit 110 has a function of formulating a combinatorial optimization problem, specifically, narrowing down candidate elements and setting an objective function and constraint conditions. The problem formulation unit 110 formulates the problem using the LLM 140. Details of the problem formulation unit 110 will be described below.

[0041] The solver 120 finds a solution to the combinatorial optimization problem formulated by the problem formulation unit 110. The result output unit 130 outputs the results to an output device 206 or transmits the results to another device via a communication interface 207.

[0042] The LLM 140 is realized by a computer executing a program, and as an example, is realized by a cloud computing system. The LLM 140 uses a large amount of text data as training data and is trained using deep learning technology to be adaptable to various natural language processing tasks. The LLM 140 performs natural language processing tasks using, for example, a deep learning model called a Transformer. Examples of the LLM 140 include GPT-4o, GPT-4, GPT-3.5, Mixtral, Mistral, LLaMA, Gemini, and Cloude3.

[0043] [Processing] Fig. 3 is a flowchart illustrating the flow of processing performed by the plan creation device 100 according to this embodiment. Fig. 4 is a flowchart showing details of the element narrowing down process S301, and Fig. 5 is a diagram illustrating interactions with the LLM 140 during this process. Fig. 6 is a flowchart showing details of the score and cost setting process S302, and Fig. 7 is a diagram illustrating interactions with the LLM 140 during this process.

[0044] In step S301, the candidate element selection unit 111 narrows down the elements. Details of this process will be described with reference to the flowchart in FIG.

[0045] In step S401, the candidate element selection unit 111 acquires user preferences. For example, the candidate element selection unit 111 acquires text or voice representing the user preferences from the user. Note that information representing the user preferences may be information on past behavioral history, such as purchase history, visit history, and web browsing history, or attribute information, such as the user's age, gender, and occupation. The information representing the user preferences may be acquired directly from the user or may be acquired from another device that stores such information.

[0046] In step S402, the candidate element selection unit 111 queries the LLM 140 for elements that match the user's preferences. When querying the LLM 140, the candidate element selection unit 111 combines the information representing the user's preferences acquired in step S401 with the prompt information 204 to generate a prompt (instruction statement) to query the LLM 140. FIG. 5A illustrates an example of a prompt. The prompt 500 includes a system prompt 501 and a user prompt 502. The system prompt 501 is predefined as the prompt information 204 and includes task definition information, which is information describing in natural language the task to be performed by the LLM 140. The prompt in FIG. 5A requests the LLM 140 to suggest tourist destination candidates that match the user's preferences, and instructs the LLM 140 to include the name, address, and reason for the recommendation. The task information illustrated in FIG. 5A is merely an example; specifying the task content in more detail allows the LLM 140 to perform the intended processing. In addition to the task definition information, the system prompt 501 may also include input / output examples, which are pairs of user inputs and corresponding output examples, and the number of input / output examples included may be one (one-shot) or two or more (few-shot).The user prompt 502 is assumed to use the text entered by the user in step S401 as is.

[0047] When the candidate element selection unit 111 sends the prompt 500 to the LLM 140, a response to the prompt 500 is obtained from the LLM 140. Fig. 5B shows an example of a response 503 from the LLM 140 to the prompt 500. The response 503 includes the name, address, and reason for the recommendation of the recommended tourist spot, as specified in the prompt 500. The candidate element selection unit 111 can parse the response 503 to obtain the landmarks of the recommended tourist spot, their addresses, and the reason for the recommendation.

[0048] In step S403, the candidate element selection unit 111 presents the elements recommended by the LLM 140 to the user. In step S404, the candidate element selection unit 111 accepts the user's selection of an element and stores the selected element. The user may view the presented elements, return to step S401, modify their preferences (user prompt), and receive suggestions of candidate elements for the modified preferences. The candidate element selection unit 111 stores the elements selected before modifying the user prompt, and adds the elements selected after modifying the prompt to the candidate elements.

[0049] FIG. 5C shows an example of a user interface for presenting recommended elements by the LLM 140 and selecting a recommended element. Since this example is for recommending tourist spots, pins representing the locations of tourist spots recommended by the LLM 140 are superimposed on a map. As an example, a white pin 504 represents an element not selected by the user, and a black pin 507 represents an element selected by the user. A frame 505 indicates an element currently focused on by the user, and an explanatory text 506 relating to the focused element is displayed. The location of the tourist spot recommended by the LLM 140, i.e., its position on the map, can be determined from the address included in the answer 503. The explanatory text 506 utilizes the reason for recommendation included in the answer 503. The user can select a tourist spot of interest on this interface.

[0050] This completes the candidate narrowing down process in step S301.

[0051] In step S302, the problem formulation unit 110 sets the score and cost of each element. Details of this process will be described with reference to the flowchart in FIG.

[0052] In step S601, the problem formulation unit 110 acquires the user's preferences and candidate elements. The problem formulation unit 110 may acquire the user's preferences again from the user, or may reuse the preferences acquired in step S401. The candidate elements are the elements selected by the user in step S404.

[0053] In step S602, the problem formulation unit 110 generates a prompt for each candidate element, inquiring about the score and cost taking into account the user's preferences, and obtains a response to the prompt from the LLM 140. When querying the LLM 140, the problem formulation unit 110 combines the information representing the user's preferences and candidate elements acquired in step S601 with the prompt information 204 to generate a prompt (instruction statement) to query the LLM 140. FIG. 7A shows an example of a prompt. The prompt 700 includes a system prompt 701 and a user prompt 702. The system prompt 701 is information predefined as the prompt information 204 and includes task definition information, which is information describing the task to be performed by the LLM 140 in natural language. The prompt in FIG. 7A requests that the satisfaction level and length of stay at each tourist destination be determined taking into account the user's preferences. In this case, as shown in the figure, by specifically specifying the ranges and calculation methods for satisfaction and cost, the LLM 140 can perform the intended processing. The system prompt 701 instructs that the satisfaction level and length of stay be determined based on the user's preferences. For example, the system prompt 701 may include determining the satisfaction level based on the degree of agreement with the user's preferences, and determining the length of stay by increasing or decreasing a typical value in consideration of the degree of agreement with the user's preferences. In the user prompt 702, the preferences are text entered by the user, and the tourist attraction list is the element selected by the user in step S404. The user's preferences may be of any kind, and may include preferences for the tourist attractions themselves as well as preferences for the entire travel plan.

[0054] 7B is a diagram showing an example of a response 703 from the LLM 140 to the prompt 700. The response 703 includes the user's satisfaction level and length of stay at each tourist attraction, as specified in the prompt 700. Note that the satisfaction level is expressed as an integer between 0 and 10, and the length of stay is expressed in minutes.

[0055] In step S603, the objective function setting unit 112 and the constraint condition setting unit 113 process the response from the LLM 140 to determine the score and cost of each candidate element.

[0056] This completes the process of setting the score and cost of each element in step S302.

[0057] In step S303, the problem formulation unit 110 formulates a problem, including candidate elements, their scores and costs, an objective function, constraints, and so forth. "Formulating a problem" here refers to expressing the problem in a format that can be processed by the solver 120. Note that when formulating the problem, additional candidate elements other than those selected by the user may be added, and information about the objective function and constraints may be acquired from the user or other information sources. The other information sources may be accessible in an open environment, such as the Internet, or in a closed environment, such as an internal company network. Information from other information sources may be acquired not only directly but also via LLM. The latter is called RAG (Retrieval Augmented Generation), and LLM is a method for generating an answer to an input prompt by combining information obtained from other information sources.

[0058] FIG. 8 is a diagram illustrating problem formulation in itinerary planning. As shown in the figure, candidate tourist destinations and their satisfaction levels (scores) and stay times (costs) are determined using the LLM 140 (reference numeral 801). The upper limit of the total stay time is determined based on user input (reference numeral 802). The travel time between tourist destinations is acquired from a map service or route guidance service (reference numeral 803). As an example, the problem formulation unit 110 defines the objective function as the sum of satisfaction levels vo, defines a constraint that the sum of the stay times vc at each tourist destination and the travel time vm be equal to or less than an upper limit Th, and defines an optimization problem as selecting from candidate elements elements that maximize the objective function under this constraint (reference numeral 804).

[0059] In step S304, the solver 120 is used to find a solution to the optimization problem formulated by the problem formulation unit 110. Then, in step S305, the result output unit 130 outputs the solution obtained from the solver 120.

[0060] Output 805 is an example of a result output by the result output unit 130. This output 805 includes, in text, tourist attractions to be visited, the duration of stay there, and the travel time to the next destination. In this example, the results are displayed in text, but any method of outputting the results may be used, such as a graphical display associated with a map display. As an example, output 805 may graphically display the travel route on a map. After viewing the results shown in output 805, the user may return to step S301 and reselect a candidate, or may modify some variables (such as the upper limit of the duration of stay) on the result display screen and return to any of steps S301 to S303 to interactively proceed from there.

[0061] [Advantageous Effects] In this embodiment, the candidate elements are narrowed down according to the user's preferences, thereby reducing calculation time. Furthermore, since the candidate elements are narrowed down based on the user's preferences, a solution that matches the user's preferences can be obtained. Furthermore, the LLM is used to narrow down the candidate elements, and the user only needs to provide their preferences in natural language (text), and the LLM presents the reasons for recommending elements, which is highly convenient for the user. Furthermore, the objective function and constraints are also determined according to the user's preferences using the LLM, which facilitates problem formulation.

[0062] [Experiment] (Experiment 1) In Experiment 1, 12 participants were asked to create a travel plan. More specifically, participants were asked to create the most satisfactory travel plan for a city they had never visited, within a travel time (total travel time and stay time) of 7 to 10 hours. Participants were asked to use three methods to plan their trip. The first method was a baseline (comparison example), in which participants freely used Google Search, Google Maps, ChatGPT-4, and a spreadsheet with a travel time input field and a setting to calculate the total. The second method used the tool of the present method (using one LLM agent). The third method used the tool of the present method with multiple LLM agents.

[0063] Participants were asked to perform the three methods and completed the following questionnaire for each. The questionnaire included a total of 11 questions categorized into three categories: overall task, search task, and route planning task. The questions were as follows, and participants were asked to rate each question on a 7-point scale, indicating whether they agreed or disagreed. 1. Overall task 1-1. I was able to create an efficient travel plan. 1-2. It was easy to create a travel plan. 1-3. Creating a travel plan was enjoyable. 2. Search task 2-1. I was able to confidently select tourist destinations from the suggested candidates or search results. 2-2. I was able to select tourist destinations that satisfied me from the suggested candidates or search results. 2-3. The suggested candidates or search results matched my preferences. 2-4. The information obtained from the suggestions or search responded flexibly to my requests. 3. Route planning task 3-1. I think the travel plan I created is a good plan. 3-2. The travel plan reflects my preferences. 3-3. The length of time spent at each tourist destination in the travel plan was appropriate. 3-4. The route of the trip plan is rational, taking into account the starting and ending points.

[0064] Figure 17 is a boxplot showing the survey results, with each question shown in the order of the comparative example, the single-agent method, and the multiple-agent method. First, a Friedman test was applied to each survey item to statistically determine whether there was a significant difference between the comparative example, single agent, and multiple agent. If the Friedman test revealed a significant difference at the 5% level, a pairwise comparison was performed. The Shaffer method was used to adjust for multiple comparisons, and a Wilcoxon test was performed as a post-hoc analysis to clarify specific differences between groups. The effect size was calculated using Cliff's delta and is denoted as d.

[0065] Item 1: Overall Work. A significant difference was observed in the efficiency of travel plan creation (item 1-1) (p=0.00183). Significant differences with large effect sizes were observed between the baseline and single-agent (p=0.0352, d=-0.590) and between the baseline and multiple-agent (p=0.0269, d=-0.639). However, no significant difference was observed between multiple-agent and single-agent (p=0.8867, d=0.090). Similarly, a significant difference was observed in the ease of creation (item 1-2) (p=0.00839). Significant differences with large effect sizes were observed between the baseline and single-agent (p=0.0317, d=-0.632) and between the baseline and multiple-agent (p=0.0416, d=-0.729). On the other hand, no significant difference was found between multiple agents and single agents (p=0.5461, d=0.042).Regarding the enjoyment of creation (items 1-3), no significant difference was found among the three methods (p=0.3281).

[0066] Item 2: Search Task. The Friedman test revealed no significant differences in the assessment of confidence in destination selection (item 2-1) and satisfaction (item 2-2) (p=0.1708 and p=0.3029, respectively). However, a significant difference was found for preference reflection (item 2-3) (p=0.0302). Comparisons revealed significant differences in large effect sizes between the baseline and single-agent setups (p=0.0238, d=-0.590) and between the baseline and multiple-agent setups (p=0.0272, d=-0.458), but no significant difference was found between multiple-agent and single-agent setups (p=0.4530, d=0.181). Furthermore, no significant differences were found for flexibility in responding to user requests (item 2-4) (p=0.4056).

[0067] Item 3: Route planning. A significant difference was observed in confidence in the created itinerary (item 3-1) (p=0.00745). Significant differences in effect size were observed between the baseline and single-agent groups (p=0.0498, d=-0.729) and between the baseline and multiple-agent groups (p=0.0342, d=-0.646). However, no significant difference was observed between multiple-agent and single-agent groups (p=0.776, d=-0.0069). No significant difference was observed in the degree of preference reflection (item 3-2) (p=0.1738). Similarly, no significant difference was observed in the appropriateness of the time spent at tourist attractions (item 3-3) (p=0.0566). A significant difference was observed in the rationality of the itinerary, taking into account the starting and ending points (item 3-4) (p=0.00626). There were significant differences in large effect sizes between baseline and single-agent (p=0.0290, d=-0.610) and between baseline and multiple-agent (p=0.0290, d=-0.701), while there was no significant difference between multiple-agent and single-agent (p=0.6029 d=-0.0972).

[0068] As described above, it was shown that the proposed method (single agent and multiple agent) was significantly superior to the comparative method (baseline) for many questions. Furthermore, no significant difference was observed between the single agent and multiple agent methods.

[0069] (Experiment 2) We quantitatively evaluated whether this method can appropriately create a route plan that satisfies the user's preferences and constraints.

[0070] Using the method described above, 12 participants were asked to create travel plans for cities they had never visited before. In this case, participants were asked to create plans using the method with one LLM agent and with multiple LLM agents. Therefore, a total of 24 plans were created. The created travel plans were then evaluated based on the following three indices: Indicator 1: The difference between the total travel time in the plan and the upper limit of the travel time in the constraints Indicator 2: The total reward, i.e., the sum of the scores that represent the degree of match between each tourist destination and the user's preferences Indicator 3: The number of tourist destinations visited

[0071] As a comparative example, we adopted a prompt engineering method that solves optimization problems using GPT-4. To enable a fair comparison, the comparative example also adopted a two-stage method similar to our method. First, in both our method and the comparative example, the parameters required for optimization are obtained from GPT-4. Information extraction from GPT-4 is the same in both methods. In our method, the optimization problem is solved by providing information obtained from GPT-4 to a solver, whereas in the comparative example, the optimization problem is solved by providing prompts to GPT-4. The prompts for solving the optimization problem in the comparative example were created with reference to Non-Patent Document 2. These prompts include the score (satisfaction level) and stay time of each POI (tourist attraction), as well as the travel time between POIs. Furthermore, as in-context example information, examples of routes that satisfy the constraints and routes that do not satisfy the constraints are included.

[0072] The evaluation results are shown in Table 1 below. Table 1 shows the average and variance of 24 samples for each of the time lag, total reward score, and number of visited POIs. As can be seen from the results, our method is superior to the comparative example in all aspects. The success rate indicates the percentage of solutions that satisfy the constraints.

[0073] A Wilcoxon test was performed to identify specific differences between our method and the comparative example. The effect size was calculated using Cliff's delta and is denoted as d. The results showed that the differences in time difference, total reward score, and number of visited POIs were statistically significant ((p=0.000183, =-0.609), (p=0.000086, d=0.4149), and (p=0.000355, d=0.4514), respectively). The difference between the total travel time and the upper limit constraint in the generated plan was limited to approximately 0.5 hours with our method, while it reached approximately 1.8 hours with the comparative example. Furthermore, the total reward score and number of visited POIs were higher with our method. The success rate indicates the percentage of the resulting route that satisfied the time constraint. Our method had a 100% success rate, while the comparative example showed a relatively high performance of 94%, but also included failures.

[0074] In this way, according to this technique, it is possible to create a route plan that is more in line with the user's preferences and more in line with the constraints.

[0075] <Modification 1> In the above description, an example has been described in which the plan creation device 100 of the present disclosure is applied to travel plan creation, but the applicable problems are not particularly limited. Below, other optimization problems to which the present disclosure can be applied will be described as examples.

[0076] (Example 1: Menu Planning) An example of an optimization problem or planning problem to which the present disclosure can be applied is menu planning that takes into account a user's nutritional constraints and satisfaction. In this example, the user's attributes, daily activity level, and other health information are first provided to the LLM 140 to receive menu recommendations. The user then selects a desired menu from the recommended menus. The objective function is the sum of satisfaction levels for each menu, and the constraint is that the daily nutritional intake must satisfy a predetermined condition. Here, the satisfaction level for each menu, the nutritional content of each menu, and the daily nutritional intake conditions are determined by querying the LLM 140, but some of these may also be determined without using the LLM 140, based on other information sources or user input.

[0077] 9A shows an example of a prompt 900 that the candidate element selection unit 111 sends to the LLM 140 when narrowing down the elements. The prompt 900 includes a system prompt 901 and a user prompt 902. The system prompt 901 includes task information describing the content of the task to be performed by the LLM 140. The user prompt 902 includes the user's attributes, exercise habits, goals, dietary preferences, and the like. In this disclosure, conditions that vary for each user when formulating a menu are referred to as user preferences. Therefore, the user prompt 902 is information that includes the user's preferences.

[0078] 9B shows an example of a response 903 from the LLM 140 to the prompt 900. As specified in the prompt 900, the response 903 includes a meal menu, nutritional information, and a reason for recommendation that is in line with the user's preferences. The candidate element selection unit 111 can parse the response 903 to obtain the recommended meal menu, its nutritional information, and the reason for recommendation.

[0079] The candidate element selection unit 111 presents the response results of the LLM 140 to the user, accepts the user's selection, and stores the selected element (meal menu).

[0080] FIG. 10A shows an example of a prompt 1000 for acquiring scores representing the degree of match between each meal menu and the user's preferences, and nutritional information for each meal menu. The scores correspond to parameters constituting the objective function, and nutritional information corresponds to parameters constituting the constraints. The prompt 1000 includes a system prompt 1001 and a user prompt 1002. The system prompt 1001 includes task information describing the content of the task to be performed by the LLM 140. This task information describes outputting a list of nutritional information for the menu, a list of scores, and a list of recommended portions. The user prompt 1002 includes the user's preferences and a meal menu selected by the user. Note that the meal menu included in the user prompt 1002 may be added by the plan creation device 100 as appropriate, in addition to the meal menu selected by the user.

[0081] 10B is a diagram illustrating an example of a response 1003 from the LLM 140 to the prompt 1000. The response 1003 includes a list of nutritional information, a list of scores, and recommended serving sizes for each meal, as specified in the prompt 1000.

[0082] The problem formulation unit 110 formulates a menu planning problem based on the information obtained from the LLM 140 in this way. Specifically, the problem is formulated as follows: the candidate elements include meal menus selected by the user, and a meal menu is selected from the candidate elements so that a score representing the user's satisfaction with each meal menu (degree of match with preferences) is highest within a range that satisfies the constraints. The constraints are that the daily intake of one or more nutrients must be within a predetermined range. The amount of nutrients contained in each meal menu is obtained from the output of the LLM 140. The desired intake range for each nutrient can also be determined by querying the LLM 140.

[0083] In this way, a menu that matches the user's preferences can be created, taking into account the user's attributes, lifestyle habits, etc. In this example, the candidate elements are narrowed down while appropriately reflecting the user's preferences, so that an appropriate solution can be provided to the user in a short processing time.

[0084] (Example 2: Work Shift Planning) Another example of an optimization problem or planning problem to which the present disclosure can be applied is the planning of employee work shifts. In this example, information about the store's work schedule and employee information are provided, and the LLM 140 is asked to propose candidate employees to be assigned to each work time slot. Furthermore, the LLM 140 is used to determine a score representing the degree to which the employee assignment to each time slot matches the requirements of the employee and the store, and a work shift schedule is developed that maximizes the total score and satisfies the constraints. While there are various methods for assigning employees to each time slot, the number of combination factors is reduced by using the LLM 140 to narrow down the candidates. Furthermore, because the scores for employee assignment are determined using the LLM 140, the burden on the user is reduced. In this way, appropriate work shifts can be easily planned.

[0085] 11A shows an example of a prompt 1100 that the candidate element selection unit 111 sends to the LLM 140 when narrowing down the elements. The prompt 1100 includes a system prompt 1101 and a user prompt 1102. The system prompt 1101 includes task information that describes the content of the task to be performed by the LLM 140. This task information describes the presentation of pairs of time slots and employees, i.e., a list of employees to be assigned to each time slot, and the reason for the recommendation. The user prompt 1102 includes information such as store working hours and information about the time periods when each employee is available to work.

[0086] 11B is a diagram showing an example of a response 1103 from the LLM 140 to the prompt 1100. The response 1103 includes candidate employee assignments for each time slot as specified in the prompt 1100 and reasons for recommending them. The candidate element selection unit 111 can parse the response 1103 to obtain the recommended employee assignments and the reasons for the recommendations.

[0087] The candidate element selection unit 111 presents the response results of the LLM 140 to the user, accepts the user's selection, and stores the selected allocation.

[0088] 12A shows an example of a prompt 1200 for obtaining a match score for each employee assignment. The prompt 1200 includes a system prompt 1201 and a user prompt 1202. The system prompt 1201 includes task information describing the content of the task to be performed by the LLM 140. This task information describes that a score should be output for the assignment of employees to each time slot, taking into consideration the user (employer) preferences and employee preferences. The user prompt 1202 includes the user's preferences for work shifts and the preferences of each employee.

[0089] 12B is a diagram illustrating an example of a response 1203 of the LLM 140 to the prompt 1200. The response 1203 includes a score representing the degree of conformance to the user and employee preferences for the assignment of employees to each time slot, as specified in the prompt 1200.

[0090] The problem formulation unit 110 formulates the work shift planning problem based on the information obtained from the LLM 140 in this way. Specifically, the problem is formulated as follows: the candidate elements include the work shift assignments selected by the user, and a meal menu is selected from the candidate elements so as to maximize the score representing the user's satisfaction with each work shift assignment (degree of match with preferences) within a range that satisfies the constraints. The constraints are that at least one employee is assigned to each time slot, and the total working hours of each employee are within a predetermined time. In this example, the problem may be formulated without using the LLM 140 to determine the constraints.

[0091] In this way, work shifts can be planned taking into account the wishes of the user (employer) and employees. In this example, the candidate elements are narrowed down while appropriately reflecting the user's preferences, so that an appropriate solution can be provided to the user in a short processing time.

[0092] (Example 3: Playlist Creation) Another example of an optimization problem or planning problem to which the present disclosure can be applied is the creation of a music playlist. In this example, information about a user's music preferences is provided to the LLM 140, which then suggests song candidates that match the user's preferences. The LLM 140 is also asked to calculate the similarity between the suggested songs and one song selected by the user (a selected song). The solver 120 then sorts the songs based on the similarity to the selected song and the features of the other songs to create a playlist. The similarity to the selected song is calculated by the LLM 140, while the other features are obtained from information sources other than the LLM 140. The solver 120 creates the playlist by solving a Hamiltonian path minimization problem.

[0093] 13 shows a user interface 1300 in this example. The request query input area 1310 includes an input area 1311 for inputting the user's preferences for the playlist in text format, a recording button 1312 for recording ambient audio, and an input area for inputting the total length of the playlist. Both the text format preferences and the recorded audio are not required; inputting either one is sufficient. When the user presses the request send button 1320, the candidate element selection unit 111 acquires the information in the request query input area 1310 as the user's preferences (S401 in FIG. 4) and queries the LLM 140 to present songs that match the user's preferences (S402).

[0094] FIG. 14A shows an example of a prompt sent by the candidate element selection unit 111 to the LLM 140 to narrow down the songs that match the user's preferences. The prompt includes a system prompt 1401 and a user prompt 1402. The system prompt 1401 includes task information describing the task that the LLM 140 is to perform. This task information describes the presentation of song candidates that match the user's preferences in order of similarity, as well as output rules. In this example, it specifies that songs that match the preferences in text format and songs similar to recorded audio are suggested. The user prompt 1402 includes text expressing the preferences entered in the input area 1311 and recorded audio data. FIG. 14B shows an example of a response 1403 from the LLM 140.

[0095] The candidate element selection unit 111 displays the contents of the response 1403 from the LLM 140 in the song candidate list 1330. Here, the candidate element selection unit 111 displays songs in descending order of matching, and if the accumulated playback time exceeds a specified time, excludes songs that follow. Note that in case the user is not satisfied with the proposed song list, the user may be able to modify the query input and make a new inquiry.

[0096] When song candidates are presented by LLM 140, the user inputs the first song in the playlist into input area 1340. It is preferable that the song selected by the user in song candidate list 1330 is copied to input area 1340. In this example, all of the presented candidate list are considered candidates for inclusion in the playlist, so the user is not required to select a song from the candidate list, but the user may be prompted to further narrow down the candidates.

[0097] When the user presses the playlist creation button 1350, the objective function setting unit 112 requests the LLM 140 to sort the song candidate list 1330 based on the similarity to the first song. Figure 14(C) shows a prompt 1404 for sorting. Here, the prompt calculates the similarity based on the song title and lyrics, but similarity may also be calculated based on other information. Figure 14(D) shows an example of the LLM 140's response to this prompt.

[0098] Once a list of song candidates sorted by similarity to the first song is obtained, the problem formulation unit 110 formulates a Hamiltonian path minimization problem taking into account similarity to the first song and indices such as the tempo and energy of the songs, and solves this problem using the solver 120. Song indices can be obtained, for example, from a song information server. The Hamiltonian path minimization problem is a problem of finding the minimum cost of a path that passes through all given elements (songs). For example, each song can be represented as a point in a three-dimensional feature space using its similarity to the first song, tempo, and energy, and the shortest Hamiltonian path problem for traveling through these points is solved. Unlike the salesman problem, this problem does not require returning to the starting point at the end.

[0099] The song list rearranged using the solver 120 is finally output to a playlist output area 1360. Although not shown in the figure, it is expected that this playlist will be used by exporting it to a music playback app, for example.

[0100] According to this example, it is possible to narrow down the candidates for songs to compose a playlist taking into consideration the user's preferences, and further to create a playlist that has small differences in similarity and other song attributes so that the transitions between songs are natural.

[0101] (Example 4: Recipe Proposal) Another example of an optimization problem or planning problem to which the present disclosure can be applied is proposing a cooking recipe. In this example, information about a user's cooking preferences is provided to the LLM 140, which then proposes several candidate dishes that match the user's preferences. Next, the user selects a candidate dish and inputs the desired number of calories. Based on this input, information about the selected recipe, such as the ingredients, the calories of each ingredient, and the priority of the ingredients in the dish, is output to the LLM 140. If the user determines that changes are necessary, such as the ingredients used, the user inquires again and modifies the ingredients used. When changes are no longer necessary, the obtained information is used to solve the optimization problem. The optimization problem is a knapsack problem in which ingredients must be selected to fit within a specified calorie limit while using high-priority ingredients.

[0102] 15A and 15B show a user interface 1500 in this example. A request query input area 1510 includes an input area 1512 for inputting the user's preference for a dish in text form, and an image input area 1511 for uploading an image of the inside of the refrigerator. When the user presses a request send button 1520, the candidate element selection unit 111 obtains the information in the request query input area 1510 as the user's preferences (S401 in FIG. 4) and queries the LLM 140 to present dishes that match the user's preferences.

[0103] FIG. 16A shows an example of a prompt sent by the candidate element selection unit 111 to the LLM 140 to output dishes that match the user's preferences. The prompt includes a system prompt 1601 and a user prompt 1602. The system prompt 1601 includes task information describing the task the LLM 140 is to perform. This task information describes the presentation of multiple dishes that match the user's preferences and the output rules. Here, the system prompt 1601 specifies that dishes that match the user's requirements are to be recommended along with the calorie content per serving and the reason for the recommendation. Furthermore, the system prompt 1601 may consider ingredients present in the refrigerator when suggesting dishes, or may suggest dishes based solely on the user's preferences. The user prompt 1602 includes text representing the preferences entered in the input area 1512 and an image of a refrigerator 1511. FIG. 16B shows an example of a response 1603 from the LLM 140. The candidate element selection unit 111 displays the information obtained from the LLM 140 in the dish candidate presentation unit 1530 based on the response 1603 .

[0104] When the LLM 140 presents candidate dishes, the user inputs the dish they actually want to eat in the text input area 1541 of the input area 1540. At this time, the user can input their preference for the dish rather than simply selecting the presented dish. In the example shown in the figure, a request for tofu hamburger with meat added has been input. The user also inputs a target calorie amount in the calorie input area 1542.

[0105] When the user presses the ingredient suggestion button 1550, the candidate element selection unit 111 requests the LLM 140 to suggest candidate ingredients for the dish. Based on the response from the LLM 140, the candidate ingredients are displayed in the candidate ingredient presentation area 1560 ( ​​FIG. 15B ). FIG. 16C shows an example of a prompt that the candidate element selection unit 111 sends to the LLM 140 to output ingredients that match the user's preferences. The prompt includes a system prompt 1604 and a user prompt 1605. The system prompt 1604 includes task information describing the task the LLM 140 is to perform. This task information describes the suggestion of candidate ingredients for the dish that match the user's preferences and calorie restrictions, as well as output rules. Here, the system requests that the priority of the ingredients used in the dish (in this example, two levels, required and optional, may be used, but three or more levels may be used) and the required amount and calories for each ingredient be output. The system also requests that an image of the refrigerator be referenced when suggesting candidate ingredients. The user prompt 1605 includes text indicating the user's preferences entered in input area 1541, the total calorie amount entered in input area 1542, and a refrigerator image 1511. Fig. 16(D) shows an example of a response 1606 from the LLM 140. The candidate element selection unit 111 displays information obtained from the LLM 140 in the ingredient candidate presentation area 1560 based on the response 1606.

[0106] If the user determines that adjustments are necessary after viewing the ingredient candidates, they can change the input content in the input area 1541 and press the ingredient suggestion button 1550 again to re-suggest the ingredients. When re-entering ingredients, for example, requests to remove a suggested ingredient (e.g., "Do not use ingredient A"), to change a suggested ingredient to another ingredient (e.g., "Please use ingredient B instead of ingredient A"), or to change the priority of an ingredient or add a required condition (e.g., "Must include ingredient A"). The LLM 140 then changes the ingredient candidates in accordance with the new request. The user repeats this process until they are satisfied with the ingredient candidates, and if they determine that no adjustments are necessary, they press the recipe suggestion button 1570.

[0107] When the recipe suggestion button 1570 is pressed, the problem formulation unit 110 formulates a combinatorial optimization problem for ingredients. The candidate combinations are the ingredient candidates proposed by the LLM 140. A knapsack problem is formulated to select, from these candidate elements, the ingredient candidate that will achieve the highest priority-based score, under the constraint that the total calorie content must be within an upper limit. A constraint that required ingredients must be used may be added. The knapsack problem formulated in this way is solved using the solver 120 to determine the ingredients to be used in the dish. The plan creation device 100 then provides the determined ingredients to the LLM 140, causing it to output a procedure for creating the dish, and outputs the ingredients to be used and the cooking procedure to the recipe display unit 1580.

[0108] According to this example, it is possible to narrow down the dishes that suit the user's preferences and the candidate ingredients to be used in those dishes, and a combinatorial optimization problem is solved using the narrowed-down ingredients, making it possible to efficiently propose recipes that suit the user's preferences.

[0109] In this example, the ingredient candidate list 1560 is displayed and the optimization problem is solved and the procedure is presented after the user's confirmation; however, the process may proceed without presenting the ingredient candidate list 1560, and the recipe may be displayed, and the user may view the resulting recipe and request changes.

[0110] <Variation 2> This variation describes a method for adapting to individual users when narrowing down candidate elements using an LLM. In this variation, multiple systems for suggesting candidate elements are prepared. The multiple suggestion systems may be realized by using different LLMs, or by providing different prompts to the same LLM to assign different characteristics. To assign different characteristics to an LLM, specifically, multiple system prompts may be prepared and queries may be made using each prompt. For example, system prompt 501 in FIG. 5 assigns the characteristic "You are a travel agent who proposes attractive travel plans to users." However, the characterization of the LLM can be changed by changing this wording.

[0111] The candidate element selection unit 111 provides prompts to multiple candidate narrowing systems (i.e., different LLMs or LLMs with different characteristics) and obtains candidate elements from each. For example, if there are N candidate narrowing systems, the candidate element selection unit 111 assigns a probability Pn (where 0≦Pn≦1, ΣPn=1) to the nth system. If the number of candidates proposed to the user is M, the probability that the proposal from the nth system will be adopted for the mth candidate is set to Pn. The probability Pn assigned to each system may be learned to maximize the number of candidate elements selected by the user. For example, the candidate element selection unit 111 uses a linear bandit algorithm to learn the selection probability Pn of each system so as to maximize the reward, with the number of candidates selected by the user being used as a reward. This enables more accurate narrowing down of candidates that match the user's preferences.

[0112] <Modification 3> The information provided as the user prompt is not limited to text data, and may include at least one of image data, video data, audio data, and sensor data (time-series data of sensor values). For example, at least one of an electroencephalogram sensor, a speed sensor, an acceleration sensor, a gyro sensor, an ultrasonic sensor, and a radiation sensor may be used as the sensor information.

[0113] <Modification 4> In the above-described embodiment, text entered by the user is transmitted to the LLM 140 as a user prompt. Therefore, if an inappropriate input is made, it is conceivable that a malfunction or an inappropriate output will occur. Therefore, before transmitting the user prompt to the LLM, the LLM 140 may be made to determine whether the user prompt contains inappropriate content, and if it is determined to be inappropriate, the processing may be terminated.

[0114] Other Embodiments The above-described embodiment is merely an example, and the present disclosure can be implemented with appropriate modifications within the scope of the gist thereof.

[0115] <Note> An information processing device (100) that determines an element from a plurality of elements that satisfies a constraint condition and optimizes an objective function, the information processing device (100) comprising: a problem setting unit (110, 111) that selects a candidate element from the plurality of elements based on an output (503) obtained by providing an input (502) from a user to a large-scale language model (140); and a solution finding unit (120) that determines, for the candidate element, an element that satisfies the constraint condition and optimizes the objective function.

[0116] 100: Information processing device 110: Problem formulation unit 111: Candidate element selection unit 112: Objective function setting unit 113: Constraint condition setting unit 120: Solver (solution finding unit) 130: Result output unit 140: LLM

Claims

1. An information processing device that determines, from among a plurality of elements, an element that satisfies constraint conditions and optimizes an objective function, comprising: a problem setting unit that selects candidate elements from among the plurality of elements based on the output obtained by providing input from a user to a large-scale language model; and a solution-finding unit that determines, with respect to the candidate elements, an element that satisfies the constraint conditions and optimizes the objective function.

2. The information processing device according to claim 1, wherein the input from the user is information expressing the user's preferences in natural language.

3. The information processing device according to claim 2, wherein the input from the user further includes at least one of image data, video data, audio data, and sensor data.

4. The information processing device described in claim 1, characterized in that the problem setting unit presents to the user tentative candidate elements selected from the plurality of elements based on the output obtained by providing input from the user to a large-scale language model, and determines the element selected by the user from the tentative candidate elements as the candidate element.

5. The information processing device according to claim 4, wherein the problem setting unit obtains text representing an explanation of the tentative candidate element from the large-scale language model and presents the text to the user.

6. The information processing device described in claim 1, characterized in that the objective function is the sum of scores assigned to each of the multiple elements, and the problem setting unit determines the score to be assigned to the candidate element based on the output obtained by providing input from the user to the large-scale language model.

7. The information processing device described in claim 6, characterized in that the problem setting unit instructs the large-scale language model to return a degree of match between the user's preferences contained in the input from the user and the candidate element, and determines the degree of match obtained from the large-scale language model as the score of the candidate element.

8. The information processing device described in claim 1, characterized in that the constraint conditions include a condition that the sum of the costs assigned to each of the multiple elements is less than or equal to a threshold, and the problem setting unit determines the cost to be assigned to the candidate element based on the output obtained by providing input from the user to the large-scale language model.

9. The information processing device according to claim 1, characterized in that the optimization is a combinatorial optimization problem, and the solution-finding unit determines a combination of elements so as to optimize the sum of the scores assigned to the selected elements while satisfying the constraint that the costs assigned to the selected elements are less than or equal to a threshold.

10. The information processing device according to claim 1, characterized in that the problem setting unit selects a provisional candidate element to be presented to the user from provisional candidate elements obtained by applying the input from the user to a plurality of large-scale language models or a plurality of large-scale language models with different characteristics, and determines an element selected by the user from the provisional candidate elements presented to the user as the candidate element.

11. The information processing device according to claim 8, characterized in that the problem setting unit learns the selection probability when selecting provisional candidate elements to be presented to the user from provisional candidate elements obtained from the plurality of large-scale language models or the plurality of large-scale language models with different characteristics, so as to maximize the number of provisional candidate elements selected by the user.

12. An information processing method performed by an information processing device to determine from among a plurality of elements an element that satisfies constraint conditions and optimizes an objective function, the information processing method comprising: a problem setting step of selecting a candidate element from among the plurality of elements based on the output obtained by providing input from a user to a large-scale language model; and a solution step of determining, with respect to the candidate elements, an element that satisfies the constraint conditions and optimizes the objective function.

13. A program for causing a computer to function as each means of the information processing device according to any one of claims 1 to 10.

14. A program for causing a computer to execute each step of the information processing method according to claim 12.

Images