Information processing apparatus, information processing method, and program

Abstract

To provide a new technique for accurately solving a combination optimization problem based on user's preference in a short calculation time.SOLUTION: An information processing device that determines a component that satisfies a constraint condition and optimizes an objective function from among a plurality of components, the information processing device comprising: a problem setting unit that selects a candidate component from among the plurality of components based on an output obtained by giving an input from a user to a large-scale language model; and a solution unit that determines a component that satisfies the constraint condition and optimizes the objective function from among the candidate components.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Combinatorial optimization problems, such as the Traveling Sales Problem (TSP) and the Orienteering Problem (OP), involve finding a solution that maximizes or minimizes an objective function under certain constraints. These problems are NP-hard, and it has been shown that as the number of combinations increases, the amount of computation increases exponentially, making it impossible to find the optimal solution within a realistic timeframe. Therefore, research and development is underway to develop approximation and heuristic methods to reduce computation time and determine the best possible solution.

[0003] Non-Patent Document 1 proposes a recommendation system based on user preferences using a large language model (LLM). LLM excels at learning a variety of tasks from huge datasets and is expected to accurately capture contextual nuances and user preferences. LLM updates its responses based on prompts from the user, leading to more appropriate recommendations and improving user satisfaction. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Jesse Harte, Wouter Zorgdrager, Panos Louridas, Asterios Katsifodimos, Dietmar Jannach, and Marios Fragkoulis. 2023. Leveraging Large Language Models for Sequential Recommendation. In Proceedings of the 17th ACM Conference on Recommender Systems (RecSys '23). Association for Computing Machinery, New York, NY, USA, 1096-1102. https: / / arxiv.org / abs / 2309.09261 [Non-patent document 2] Zhen Yang, Ming Ding, Qingsong Lv, Zhihuan Jiang, Zehai He, Yuyi Guo, Jinfeng Bai, and Jie Tang. 2023. Gpt can solve mathematical problems without a calculator.arXiv preprint arXiv:2309.03241 (2023). https: / / arxiv.org / abs / 2309.03241 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the recommendations or optimizations made by LLM do not always produce the desired results as intended. Solutions to improve the accuracy of optimization have been explored, but satisfactory results have not been achieved.

[0006] In consideration of the above circumstances, an object of the present disclosure is to provide a novel technique for solving combinatorial optimization problems based on user preferences with high accuracy in a short calculation time. [Means for solving the problem]

[0007] One aspect of the present disclosure is an information processing device that determines, from among a plurality of elements, an element that satisfies constraint conditions and optimizes an objective function, and that includes: a problem setting unit that selects candidate elements from among the plurality of elements based on an output obtained by providing input from a user to a large-scale language model; and a solution unit that determines, for the candidate elements, an element that satisfies the constraint conditions and optimizes the objective function.

[0008] In this way, by narrowing down the elements using a large-scale language model, the calculation time can be reduced, and by narrowing down the elements based on the output of a large-scale language model in response to input from the user, the user can This makes it possible to reflect the user's preferences, and to provide the user with an appropriate solution that matches their preferences.

[0009] In one embodiment, the problem setting unit may present to the user tentative candidate elements selected from the plurality of elements based on an output obtained by providing an input from the user to a large-scale language model, and determine an element selected by the user from the tentative candidate elements as the candidate element. In this way, user selection of candidates more reliably reflects the user's preferences, and since the tentative candidate elements are provided by the large-scale language model, the burden on the user is reduced. Note that in other embodiments, the problem setting unit may determine elements output by the large-scale language model as candidate elements as they are, or may add elements obtained by other methods to the candidate elements.

[0010] In one embodiment, the problem setter may obtain text describing the tentative candidate elements from the large-scale language model and present the text to the user. For example, when instructing the large-scale language model to return the tentative candidate elements, the problem setter may instruct the large-scale language model to also return text describing the explanation. By presenting the explanation text to the user, the user can determine whether the tentative candidate elements match their preferences, and the user's preferences can be more accurately reflected in the determination of candidate elements.

[0011] In one embodiment, the objective function may be a sum of scores assigned to each of the multiple elements, and the problem setter may determine the score to be assigned to the candidate element based on an output obtained by providing the user's input to the large-scale language model. For example, the problem setter may instruct the large-scale language model to return a degree of match between the user's preferences contained in the user's input and the candidate element, and the degree of match obtained from the large-scale language model may be determined as the score of the candidate element. Note that it is not necessary to determine the entire score based on the output of the large-scale language model; part of the score may be determined based on user input, information from other sources, preset setting information, etc. Formulating a problem based on user preferences is generally time-consuming, but formulating the objective function based on the output of the large-scale language model enables simple processing.

[0012] In one embodiment, the constraints may include a condition that the sum of the costs assigned to each of the plurality of elements is equal to or less than a threshold, and the problem formulation unit may determine the cost to be assigned to the candidate element based on an output obtained by providing the user's input to the large-scale language model. For example, the problem formulation unit may instruct the large-scale language model to return the cost of the candidate element, and the output from the large-scale language model may be determined as the cost. Note that it is not necessary to determine all of the cost based on the output of the large-scale language model; some of the cost may be determined based on user input, information from other sources, preset configuration information, etc. Formulating a problem according to user preferences is generally time-consuming, but formulating constraints based on the output of the large-scale language model enables simple processing.

[0013] The optimization problem to which this aspect can be applied is any combinatorial optimization problem, and examples include travel planning, menu planning, and work shift planning.

[0014] Large Language Models (LLMs) are a type of machine learning that is also known as generative AI. A large-scale language model is a learning model that has the learning ability to generate versatile content. A large-scale language model refers to a language model that has increased scale in at least one of the amount of calculation, the amount of data, and the number of model parameters. In this disclosure, a language model with 5 billion or more model parameters is called a large-scale language model. Input from a user can be in any form, such as natural language information or image information, and more specifically, it can be text, voice, or graphics. This includes information expressed as gestures, images, and any combination thereof.

[0015] The present disclosure can be understood as an information processing device or optimization device including at least some of the above means. The present disclosure can also be understood as a method including at least some of the above processing, or a program for realizing such a method, or a recording medium on which such a program is non-temporarily recorded. Furthermore, the present disclosure includes an apparatus or method in which each of the above means and processing is combined with each other to the extent possible. [Effects of the Invention]

[0016] According to the present disclosure, it becomes possible to solve combinatorial optimization problems based on user preferences with high accuracy in a short calculation time. [Brief explanation of the drawings]

[0017] [Figure 1] FIG. 1 is a diagram showing an overview of an information processing device according to the present disclosure. [Figure 2] FIG. 2 is a diagram showing a hardware configuration of a control device according to the present disclosure. [Figure 3] 1 is a flowchart showing an overall flow executed by an information processing device according to the present disclosure. [Figure 4] 10 is a flowchart showing details of an element narrowing down process in the present disclosure. [Figure 5] 10A and 10B are diagrams illustrating examples of prompts and LLM answers in the element narrowing down process in the present disclosure. [Figure 6] 10 is a flowchart showing details of a score / cost setting process in the present disclosure. [Figure 7] FIG. 10 illustrates an example of prompts and LLM answers in the score-cost setting process of the present disclosure. [Figure 8] FIG. 1 is a diagram illustrating a problem formulation in the present disclosure. [Figure 9] 10A and 10B are diagrams showing examples of prompts and LLM answers in the element narrowing down process in Example 1 of Modification 1 of the present disclosure. [Figure 10] A figure showing examples of prompts and LLM answers in score-cost processing in example 1 of variant example 1 of the present disclosure. [Figure 11] 10A and 10B are diagrams showing examples of prompts and LLM answers in the element narrowing down process in Example 2 of Modification Example 1 of the present disclosure. [Figure 12] A figure showing examples of prompts and LLM answers in the score and cost setting process in example 2 of variant example 1 of the present disclosure. [Figure 13] 10A and 10B are diagrams showing examples of prompts and LLM answers in the element narrowing down process in Example 3 of Variation 1 of the present disclosure. [Figure 14] A figure showing examples of prompts and LLM answers in the score and cost setting process in example 3 of variant example 1 of the present disclosure. [Figure 15A] FIG. 10 is a diagram showing examples of prompts and LLM answers in the element narrowing down process in Example 4 of Variation 1 of the present disclosure. [Figure 15B] FIG. 10 is a diagram showing examples of prompts and LLM answers in the element narrowing down process in Example 4 of Variation 1 of the present disclosure. [Figure 16] A figure showing examples of prompts and LLM answers in the score and cost setting process in example 4 of variant example 1 of the present disclosure. [Figure 17] FIG. 10 is a diagram showing the results of a user questionnaire survey in the evaluation of travel planning according to the present disclosure (Experiment 1). DETAILED DESCRIPTION OF THE INVENTION

[0018] <Application example> First, an example of a plan creation device to which the present disclosure is applied will be described. The plan creation device creates a plan that matches the user's preferences, taking into account the user's preferences. For example, the plan creation device solves optimization problems such as travel plans, menu planning, and work shift plans, and creates a plan that matches the user's preferences by setting the objective function and constraint conditions according to the user's preferences. Complete.

[0019] 1 is a diagram illustrating an overview of a plan creation device 100 according to the present disclosure. The plan creation device 100 has, as functional units, a problem formulation unit 110, a solver 120, and a result output unit 130. The problem formulation unit 110 further has a candidate element selection unit 111, an objective function setting unit 112, and a constraint condition setting unit 113.

[0020] The problem formulation unit 110 has a function of formulating a combinatorial optimization problem, specifically, narrowing down candidate elements and setting objective functions and constraints. The problem formulation unit 110 formulates the problem using a large-scale language model (LLM) 140.

[0021] The candidate element selection unit 111 narrows down the elements using the LLM 140. For example, the candidate element selection unit 111 provides the LLM 140 with a prompt including the user's preferences, receives from the LLM 140 suggestions of elements that match the user's preferences, and determines elements (candidate elements) to be used for the optimization problem based on the suggestions from the LLM 140. For example, the candidate element selection unit 111 may present suggestions from the LLM 140 to the user and select elements selected by the user from the suggestions as candidate elements, or the candidate element selection unit 111 may determine candidate elements from the suggestions from the LLM 140 based on predetermined criteria, or may simply determine the suggestions from the LLM 140 as candidate elements. Note that the candidate element selection unit 111 may query multiple different LLMs 140, or query the same LLM 140 after assigning different characteristics to it, or may suggest elements to the user based on multiple answers obtained in this manner.

[0022] The objective function setting unit 112 determines the score (parameters constituting the objective function) to be assigned to each element using the LLM 140. For example, the objective function setting unit 112 instructs the LLM 140 to return the degree of match between the candidate element and the user's preference, and determines the degree of match obtained from the LLM 140 as the score of the candidate element.

[0023] The constraint setting unit 113 determines the cost (parameters constituting the constraint) to be assigned to each element or a combination of elements using the LLM 140. For example, the constraint setting unit 113 instructs the LLM 140 to return the cost related to the selection of a candidate element, and determines the cost obtained from the LLM 140 as the cost of the candidate element.

[0024] The candidate element selection unit 111, objective function setting unit 112, and constraint condition setting unit 113 make it possible to determine (narrow down) the elements that make up a combinatorial optimization problem and to formulate the objective function and constraint conditions. After formulating the combinatorial optimization problem, the problem formulation unit 110 provides the formula to the solver 120 to obtain a solution to the problem.

[0025] The solver 120 is a functional unit that solves combinatorial optimization problems and is configured using any existing algorithm. The algorithm adopted by the solver 120 can be determined appropriately depending on the type of optimization problem to be solved, and examples that can be adopted include the branch and bound method, dynamic programming, local search method, and Edmund's method.

[0026] The result output unit 130 outputs the optimal solution found by the solver 120 in the form of display on a display device, storage in a storage device, communication to another device, or the like.

[0027] An example of a problem that the plan creation device 100 handles is travel planning. In travel planning, for example, tourist destinations (visiting destinations) and the order of visiting them are determined so that the total travel and stay time meets predetermined constraints and the user's satisfaction is maximized. In this example, the candidate element selection unit 111 provides the user's preferences, such as the user's interests and desired conditions for tourist destinations, to the LLM 140, and the LLM 140 presents candidate tourist destinations that match the user's preferences. The candidate element selection unit 111, for example, presents tourist destinations obtained from the LLM 140 to the user and determines the tourist destination selected by the user as a candidate element. The objective function setting unit 112 provides the user's preferences to the LLM 140 and causes it to output the user's satisfaction (score) with respect to the candidate elements. The constraint condition setting unit 113 queries the LLM 140 about the stay time at each tourist destination and determines the stay time as a cost. Note that travel time between tourist destinations is also a cost, but the travel time is acquired from another information source. The upper limit of travel time (the sum of stay time and travel time) may be determined using a value input by the user, or this may also be determined using the LLM 140.

[0028] In this way, narrowing down the candidate elements according to the user's preferences allows for a solution that matches the user's preferences, and limiting the number of elements reduces the calculation time. Furthermore, the user's preferences are reflected in the scores that make up the objective function and the costs that make up the constraints, so a solution that matches the user's preferences can be obtained. Furthermore, communication with the LLM140 can be done using natural language (text), making it easy and convenient for the user.

[0029] <Embodiment> [composition] Fig. 2 schematically illustrates an example of the hardware configuration of the plan creation device 100 according to this embodiment. As illustrated in Fig. 2, the plan creation device 100 according to this embodiment is a computer (information processing device) in which a control unit 201, a storage unit 202, an input device 205, an output device 206, a communication interface 207, and a drive 208 are electrically connected.

[0030] The control unit 201 includes a CPU (Central Processing Unit) which is a hardware processor. The control unit 201 includes a RAM (Random Access Memory), a ROM (Read Only Memory), etc., and is configured to execute information processing based on programs and various data. The control unit 201 (CPU) is an example of a processor resource.

[0031] The storage unit 202 is an example of a memory resource, and is configured, for example, by a hard disk drive, a solid state drive, etc. In this embodiment, the storage unit 202 stores various information such as a plan creation program 203, prompt information 204, etc.

[0032] The planning program 203 is a program for causing the planning device 100 to execute information processing for a combinatorial optimization problem. The planning program 203 includes a series of instructions for the information processing. The prompt information 204 is information to be included in the instruction statement (prompt) 140 that queries the LLM 140, and corresponds to a system prompt, which will be described later.

[0033] The input device 205 is a device for inputting, such as a mouse, keyboard, keypad, or touch panel device. The output device 206 is a device for outputting, such as a display or speaker. A user can operate the plan creation device 100 by using the input device 205 and the output device 206. The input device 205 and the output device 206 may be integrated into one device, such as a touch panel display.

[0034] The communication interface 207 is, for example, a wired LAN (Local Area Network) module, a wireless LAN module, etc., and is an interface for performing wired or wireless communication via a network. The plan creation device 100 can perform data communication with the LLM 140 and other computers via the communication interface 207.

[0035] The drive 208 is, for example, a CD drive, a DVD drive, or the like, and is a drive device for reading various information such as programs stored in a storage medium 209. At least one of the generation program 203 and the prompt information 204 may be stored in a storage medium 209 .

[0036] The storage medium 209 is a medium that stores information such as programs electrically, magnetically, optically, mechanically, or chemically so that a computer or other device, machine, etc. can read the stored information. The plan creation device 100 may acquire at least one of the plan creation program 203 and the prompt information 204 from the storage medium 209.

[0037] 2 illustrates a disk-type storage medium such as a CD or DVD as an example of the storage medium 209. However, the type of the storage medium 209 is not limited to a disk type, and may be a type other than a disk. Examples of storage media other than a disk type include semiconductor memories such as flash memories. The type of the drive 208 may be selected appropriately depending on the type of the storage medium 209.

[0038] Regarding the specific hardware configuration of the plan creation device 100, components can be omitted, replaced, or added as appropriate depending on the embodiment. For example, the control unit 201 may include multiple hardware processors. The hardware processors may be a microprocessor, a field-programmable gate array (FPGA), a digital signal processor (DSP), etc. The storage unit 202 may be configured with RAM and ROM included in the control unit 201. At least one of the input device 205, the output device 206, the communication interface 207, and the drive 208 may be omitted. The plan creation device 100 may be configured with multiple computers. In this case, the hardware configurations of the computers may or may not be the same. Furthermore, the plan creation device 100 may be an information processing device designed specifically for the services to be provided, as well as a general-purpose server device, a general-purpose PC (Personal Computer), an industrial PC, etc.

[0039] The plan creation device 100 functions as a problem formulation unit 110, a solver (solution unit) 120, and a result output unit 130 shown in Fig. 1 when the control unit 201 executes the plan creation program 203. The problem formulation unit 110 further includes a candidate element selection unit 111, an objective function setting unit 112, and a constraint condition setting unit 113.

[0040] The problem formulation unit 110 has a function of formulating a combinatorial optimization problem, specifically, narrowing down candidate elements and setting an objective function and constraint conditions. The problem formulation unit 110 formulates the problem using the LLM 140. Details of the problem formulation unit 110 will be described below.

[0041] The solver 120 finds a solution to the combinatorial optimization problem formulated by the problem formulation unit 110. The result output unit 130 outputs the results to an output device 206 or transmits the results to another device via a communication interface 207.

[0042] LLM140 is implemented by a computer executing a program, for example, a cloud computing system. LLM140 uses large amounts of text data as training data and is trained using deep learning techniques to be adaptable to various natural language processing tasks. LLM140 performs natural language processing tasks using a deep learning model called a Transformer, for example. Examples of LLM140 include GPT-4o, GPT-4, GPT-3.5, Mixtral, Mistral, LLaMA, Gemini, and Claude3.

[0043] [process] Fig. 3 is a flowchart illustrating the flow of processing performed by the plan creation device 100 according to this embodiment. Fig. 4 is a flowchart showing details of the element narrowing down process S301, and Fig. 5 is a diagram illustrating interaction with the LLM 140 during this process. Fig. 6 is a flowchart showing details of the score and cost setting process S302, and Fig. 7 is a diagram illustrating interaction with the LLM 140 during this process.

[0044] In step S301, the candidate element selection unit 111 narrows down the elements. Details of this process will be described with reference to the flowchart in FIG.

[0045] In step S401, the candidate element selection unit 111 acquires user preferences. For example, the candidate element selection unit 111 acquires text or voice representing the user preferences from the user. Note that information representing the user preferences may be information on past behavioral history such as purchase history, visit history, and web browsing history, or attribute information such as the user's age, gender, and occupation. The information representing the user preferences may be acquired directly from the user or may be acquired from another device that stores such information.

[0046] In step S402, the candidate element selection unit 111 queries the LLM 140 for elements that match the user's preferences. When querying the LLM 140, the candidate element selection unit 111 combines the information representing the user's preferences acquired in step S401 with the prompt information 204 to generate a prompt (instruction statement) to query the LLM 140. FIG. 5A illustrates an example of a prompt. The prompt 500 includes a system prompt 501 and a user prompt 502. The system prompt 501 is predefined as the prompt information 204 and includes task definition information, which is information describing in natural language the task to be performed by the LLM 140. The prompt in FIG. 5A requests the LLM 140 to suggest tourist destination candidates that match the user's preferences, and instructs the LLM 140 to include the name, address, and reason for recommendation of the tourist destination. The task information illustrated in FIG. 5A is merely an example; specifying the task content in more detail allows the LLM 140 to perform the intended processing. In addition to the task definition information, the system prompt 501 may also include input / output examples, which are pairs of user inputs and corresponding output examples, and the number of input / output examples included may be one (one-shot) or two or more (few-shot).The user prompt 502 is assumed to use the text entered by the user in step S401 as is.

[0047] When the candidate element selection unit 111 sends a prompt 500 to the LLM 140, a response is obtained from the LLM 140. Figure 5(B) shows an example of a response 503 from the LLM 140 to the prompt 500. The response 503 includes the name, address, and reason for recommendation of the recommended tourist destination, as specified in the prompt 500. By parsing the response 503, the candidate element selection unit 111 can obtain the landmarks of the recommended tourist destination, their addresses, and the reason for recommendation.

[0048] In step S403, the candidate element selection unit 111 presents the elements recommended by the LLM 140 to the user. In step S404, the candidate element selection unit 111 accepts the user's selection of an element and stores the selected element. The user may view the presented elements, return to step S401, modify their preferences (user prompt), and receive suggestions of candidate elements for the modified preferences. The candidate element selection unit 111 stores the elements selected before modifying the user prompt, and adds the elements selected after modifying the prompt to the candidate elements.

[0049] FIG. 5(C) shows an example of a user interface for presenting and selecting recommended elements by the LLM 140. Since this example is a recommendation of tourist spots, the LLM 140 Pins representing the locations of tourist attractions recommended by the LLM 140 are superimposed on the map. As an example, a white pin 504 represents an element that has not been selected by the user, and a black pin 507 represents an element that has been selected by the user. A frame 505 indicates the element that the user is currently focusing on, and an explanatory text 506 regarding the focused element is displayed. The location of the tourist attraction recommended by the LLM 140, i.e., its position on the map, can be determined from the address included in the answer 503. The explanation text 506 uses the reason for recommendation included in the answer 503. The user can select a tourist attraction of interest on this interface.

[0050] This completes the candidate narrowing down process in step S301.

[0051] In step S302, the problem formulation unit 110 sets the score and cost of each element. Details of this process will be described with reference to the flowchart in FIG.

[0052] In step S601, the problem formulation unit 110 acquires the user's preferences and candidate elements. The problem formulation unit 110 may acquire the user's preferences again from the user, or may reuse the preferences acquired in step S401. The candidate elements are the elements selected by the user in step S404.

[0053] In step S602, the problem formulation unit 110 generates a prompt for each candidate element, inquiring about the score and cost taking into account the user's preferences, and obtains a response to the prompt from the LLM 140. When querying the LLM 140, the problem formulation unit 110 combines the information representing the user's preferences and candidate elements acquired in step S601 with the prompt information 204 to generate a prompt (instruction statement) to be sent to the LLM 140. FIG. 7A shows an example of a prompt. The prompt 700 includes a system prompt 701 and a user prompt 702. The system prompt 701 is predefined as the prompt information 204 and includes task definition information, which is information describing the task to be performed by the LLM 140 in natural language. The prompt in FIG. 7A requests that the satisfaction level and length of stay at each tourist destination be determined taking into account the user's preferences. In this case, as shown in the figure, the value ranges and calculation methods for satisfaction and cost can be specifically specified, allowing the LLM 140 to perform the intended processing. System prompt 701 instructs that at least the satisfaction level and length of stay be determined based on the user's preferences, and may include, for example, determining satisfaction based on the degree of match with the user's preferences, and determining length of stay by increasing or decreasing a typical value in consideration of the degree of match with the user's preferences. In user prompt 702, the preferences are text entered by the user, and the tourist attraction list is the element selected by the user in step S404. The user's preferences may be of any kind, and may include preferences for the tourist attractions themselves as well as preferences for the entire travel plan.

[0054] 7B is a diagram showing an example of a response 703 from the LLM 140 to the prompt 700. The response 803 includes the user's satisfaction level and length of stay at each tourist attraction, as specified in the prompt 700. Note that the satisfaction level is expressed as an integer between 0 and 10, and the length of stay is expressed in minutes.

[0055] In step S603, the objective function setting unit 112 and the constraint condition setting unit 113 process the response from the LLM 140 to determine the score and cost of each candidate element.

[0056] This completes the process of setting the score and cost of each element in step S302.

[0057] In step S303, the problem formulation unit 110 selects candidate elements, The problem is formulated, including scores, costs, objective functions, constraints, etc. "Formulating a problem" here refers to expressing the problem in a format that can be processed by the solver 120. When formulating the problem, it is possible to add candidate elements other than those selected by the user, and obtain information about the objective function and constraints from the user or other information sources. Other information sources may be accessible in an open environment, such as the Internet, or in a closed environment, such as an internal company network. Information from other information sources may be obtained directly or via LLM. The latter is called RAG (Retrieval Augmented Generation), and LLM is a method for generating an answer to an input prompt by combining information obtained from other information sources.

[0058] FIG. 8 is a diagram illustrating problem formulation in itinerary planning. As shown in the figure, candidate tourist destinations and their respective satisfaction levels (scores) and stay times (costs) are determined using the LLM 140 (reference numeral 801). An upper limit on the total stay time is determined based on user input (reference numeral 802). The travel time between tourist destinations is acquired from a map service or route guidance service (reference numeral 803). As an example, the problem formulation unit 110 defines the objective function as the sum of satisfaction levels vo, defines a constraint that the sum of the stay times vc at each tourist destination and the travel time vm be equal to or less than the upper limit Th, and defines an optimization problem as the problem of selecting from candidate elements those elements that maximize the objective function under this constraint (reference numeral 804).

[0059] In step S304, the solver 120 is used to find a solution to the optimization problem formulated by the problem formulation unit 110. Then, in step S305, the result output unit 130 outputs the solution obtained from the solver 120.

[0060] Output 805 is an example of a result output by result output unit 130. This output 805 includes, in text, tourist attractions to be visited, the duration of stay there, and the travel time to the next destination. In this example, the results are displayed in text, but any method of outputting the results may be used, such as a graphical display associated with a map display. As an example, output 805 may display the travel route graphically on a map. After viewing the results shown in output 805, the user may return to step S301 and reselect a candidate, or may modify some variables (such as the upper limit of the duration of stay) on the result display screen and return to any of steps S301 to S303 to interactively proceed from there.

[0061] [Beneficial Effects] In this embodiment, the candidate elements are narrowed down according to the user's preferences, thereby reducing calculation time. Furthermore, since the candidate elements are narrowed down based on the user's preferences, a solution that matches the user's preferences can be obtained. Furthermore, LLM is used to narrow down the candidate elements, and the user only needs to provide their preferences in natural language (text), and the LLM presents the reasons for recommending elements, making it highly convenient for the user. Furthermore, the objective function and constraints are also determined according to the user's preferences using LLM, making it easier to formulate the problem.

[0062] [experiment] (Experiment 1) In Experiment 1, 12 participants were asked to create a travel plan. More specifically, they were asked to create the most satisfying travel plan for a city they had never visited before, within a travel time (total travel time and stay time) set at 7 to 10 hours. Participants were asked to use three methods to plan their trip. The first method was the baseline (comparison example), and used Google Search, Google Maps, ChatGPT-4, and Travel In this method, participants freely used a spreadsheet with time input fields and a setting that calculated the total. The second method used the tool of this method (with one LLM agent). The third method used the tool of this method with multiple LLM agents.

[0063] Participants were asked to perform the three methods and were then given the following questionnaire for each. The questionnaire included a total of 11 questions categorized into three categories: overall task, search task, and route planning task. The questions were as follows, and participants were asked to agree or disagree with each question on a 7-point scale. 1. Overall work 1-1. I was able to create an efficient travel plan 1-2. It was easy to create a travel plan 1-3. It was fun to create a travel plan 2.Exploration work 2-1. I was able to confidently select a tourist destination from the suggested candidates or search results. 2-2. I was able to select a satisfactory tourist destination from the suggested candidates or search results. 2-3. The suggestions or search results match my preferences 2-4. The information obtained through suggestions or searches responded flexibly to my requests. 3. Route planning 3-1. I think the travel plan I created is a good plan. 3-2. Travel plans reflect your preferences 3-3. The length of time spent at each tourist spot in the travel plan is appropriate 3-4. The route of the travel plan is rational, taking into account the starting and ending points.

[0064] Figure 17 is a box plot showing the survey results, with each question shown in the order of the comparative example, the method with a single agent, and the method with multiple agents. First, the Friedman test was applied to each survey item to statistically determine whether there was a significant difference between the comparative example, the single agent, and the multiple agents. If a significant difference was found at the 5% level in the Friedman test, paired comparisons were performed. The Shaffer method was used to adjust for multiple comparisons. The Wilcoxon test was used as a post-hoc analysis to reveal specific differences between groups. Effect sizes were calculated using Cliff's delta and are denoted as d.

[0065] Item 1: Overall work. A significant difference was observed in the efficiency of travel planning (item 1-1) (p=0.00183). Significant differences with large effect sizes were observed between the baseline and single agent (p=0.0352, d=-0.590) and between the baseline and multiple agents (p=0.0269, d=-0.639). However, there was no significant difference between multiple agents and a single agent (p=0.8867, d=0.090). Similarly, there was a significant difference in ease of creation (items 1-2) (p=0.00839). There was a significant difference in effect size between single-agent and multi-agent (p=0.0416, d=-0.729), while there was no significant difference between multi-agent and single-agent (p=0.5461, d=0.042). Regarding the enjoyment of creating (items 1-3), no significant difference was found among the three methods (p=0.3281).

[0066] Item 2: Search task. The Friedman test did not reveal any significant differences in the assessment of confidence in the choice of tourist destination (item 2-1) and satisfaction (item 2-2) (p=0.1708, p=0.3029, respectively). On the other hand, a significant difference was found in the reflection of preferences (item 2-3) (p=0.0302). As a result of the comparison, significant differences with large effect sizes were found between the baseline and single agent (p=0.0238, d=-0.590) and between the baseline and multiple agents (p=0.0272, d=-0.458), but not between multiple agents and single agent (p=0.4530, d=0.181). There was no significant difference in the items 2-4 regarding flexibility in responding to user requests (p=0.4056).

[0067] Item 3: Route planning. There was a significant difference in the confidence level for the created travel plan (item 3-1). A significant difference was observed between baseline and single agent (p=0.0498, d=-0.729). ) and between the baseline and multiple agents (p=0.0342, d=-0.646), there was a significant difference in effect size. On the other hand, there was no significant difference between multiple agents and a single agent (p=0.776, d=-0.0069). There was no significant difference in the degree of preference reflection (item 3-2) (p=0.1738). Similarly, there was no significant difference in the appropriateness of the time spent at tourist spots (item 3-3) (p=0.0566). There was a significant difference in the rationality of the travel plan taking into account the starting point and the destination (item 3-4) (p=0.00626). Baseline There were significant differences in large effect sizes between baseline and single agent (p=0.0290, d=-0.610) and between baseline and multiple agents (p=0.0290, d=-0.701), while there was no significant difference between multiple agents and single agent (p=0.6029 d=-0.0972).

[0068] As described above, it was shown that the proposed method (single agent and multiple agent) was significantly superior to the comparative method (baseline) for many questions. Furthermore, no significant difference was observed between the single agent and multiple agent methods.

[0069] (Experiment 2) We quantitatively evaluated whether this method can appropriately generate route plans that satisfy the user's preferences and constraints.

[0070] Using the method described above, 12 participants were asked to create travel plans for cities they had never visited before. They created plans using the method with one LLM agent and with multiple LLM agents. A total of 24 plans were created. The created travel plans were then evaluated based on the following three indicators: Indicator 1: Difference between the total travel time in the plan and the upper limit of the travel time in the constraints Indicator 2: Total reward, i.e., the sum of scores that represent the degree of match with user preferences for each tourist destination. Indicator 3: Number of tourist destinations visited

[0071] As a comparative example, we adopted a prompt engineering method that uses GPT-4 to solve optimization problems. To enable a fair comparison, the comparative example also adopted the same two-stage method as our method. First, in both our method and the comparative example, the parameters required for optimization are obtained from GPT-4. Information extraction from GPT-4 is the same in both methods. In our method, the information obtained from GPT-4 is provided to a solver to solve the optimization problem, while in the comparative example, the optimization problem is solved by providing prompts to GPT-4. The prompts used to solve the optimization problem in the comparative example were created with reference to Non-Patent Document 2. These prompts include the score (satisfaction) and stay time of each POI (tourist attraction), as well as the travel time between POIs. Furthermore, as in-context example information, examples of routes that satisfy the constraints and routes that do not satisfy the constraints are included.

[0072] The evaluation results are shown in Table 1 below. Table 1 shows the average and variance of 24 samples for time lag, total reward score, and number of visited POIs. As can be seen from the results, our method outperforms the comparative method in all aspects. The success rate indicates the percentage of solutions that satisfy the constraints. [Table 1]

[0073] A Wilcoxon test was conducted to identify specific differences between this method and the comparative example. The effect size was calculated using Cliff's delta and is denoted as d. As a result, the time lag, total reward score, and visit The difference in the number of problem POIs was statistically significant ((p=0.000183, =-0.609), (p=0.000086, d=0.4149), (p=0.000355, d=0.4514), respectively). The difference between the total travel time and the upper limit constraint in the generated plan was kept to approximately 0.5 hours by our method, while it reached approximately 1.8 hours in the comparative example. Furthermore, the total reward score and number of visited POIs were higher for our method. The success rate indicates the percentage of the obtained route that satisfied the time constraint. Our method had a success rate of 100%, while the comparative example also showed a relatively high performance of 94%, but also included failures.

[0074] In this way, according to this technique, it is possible to create a route plan that is more in line with the user's preferences and more in line with the constraints.

[0075] <Variation 1> In the above description, an example was explained in which the plan creation device 100 of the present disclosure is applied to travel plan creation, but the applicable problems are not particularly limited. Below, other optimization problems to which the present disclosure can be applied will be described as examples.

[0076] (Example 1: Menu planning) An example of an optimization problem or planning problem to which the present disclosure can be applied is menu planning that takes into account a user's nutritional constraints and satisfaction. In this example, the user first provides the LLM 140 with the user's attributes, daily activity level, and other health information to receive menu recommendations. The user then selects a desired menu from the recommended menus. The objective function is the sum of satisfaction levels for each menu, and the constraint is that the daily nutritional intake must satisfy a predetermined condition. Here, the satisfaction level for each menu, the nutritional content of each menu, and the daily nutritional intake conditions are determined by querying the LLM 140, but some of these may also be determined without using the LLM 140, based on other information sources or user input.

[0077] FIG. 9A shows an example of a prompt 900 that the candidate element selection unit 111 sends to the LLM 140 when narrowing down the elements. The prompt 900 includes a system prompt 901 and a user prompt 902. The system prompt 901 includes task information that describes the content of the task to be performed by the LLM 140. The user prompt 902 includes the user's attributes, exercise habits, goals, dietary preferences, and the like. In this disclosure, conditions that vary for each user when formulating a menu are referred to as user preferences. Therefore, the user prompt 902 is information that includes the user's preferences.

[0078] 9(B) shows an example of a response 903 from the LLM 140 to the prompt 900. As specified in the prompt 900, the response 903 includes a meal menu that matches the user's preferences, nutritional information, and a reason for recommendation. The candidate element selection unit 111 can parse the response 903 to obtain the recommended meal menu, its nutritional information, and the reason for recommendation.

[0079] The candidate element selection unit 111 presents the response results of the LLM 140 to the user, accepts the selection by the user, and stores the selected element (meal menu).

[0080] FIG. 10A shows an example of a prompt 1000 for acquiring scores representing the degree of match between each meal menu and the user's preferences, and nutritional information for each meal menu. The scores correspond to parameters constituting the objective function, and nutritional information corresponds to parameters constituting the constraints. The prompt 1000 includes a system prompt 1001 and a user prompt 1002. The system prompt 1001 includes task information describing the content of the task to be performed by the LLM 140. This task information describes outputting a list of nutritional information for the menu, a list of scores, and a list of recommended portions. The user prompt 1002 includes the user's preferences and a meal menu selected by the user. Note that the meal menus included in the user prompt 1002 may be added by the plan creation device 100 as appropriate, in addition to those selected by the user.

[0081] 10(B) is a diagram showing an example of a response 1003 of the LLM 140 to the prompt 1000. The response 1003 includes a list of nutritional information, a list of scores, and recommended serving sizes for each meal menu item, as specified in the prompt 1000.

[0082] The problem formulation unit 110 formulates the menu planning problem based on the information obtained from the LLM 140 in this way. Specifically, the problem is formulated as follows: the meal menu selected by the user is included in the candidate elements, and a meal menu is selected from the candidate elements so that the score representing the user's satisfaction with each meal menu (degree of match with preferences) is highest within a range that satisfies the constraints. The constraints are that the daily intake of one or more nutrients is within a predetermined range. The amount of nutrients contained in each meal menu is obtained from the output of the LLM 140. The desired intake range for each nutrient can also be determined by querying the LLM 140.

[0083] In this way, a menu that matches the user's preferences can be created, taking into account the user's attributes, lifestyle habits, etc. In this example, the candidate elements are narrowed down while appropriately reflecting the user's preferences, so that an appropriate solution can be provided to the user in a short processing time.

[0084] (Example 2: Work shift planning) Another example of an optimization problem or planning problem to which the present disclosure can be applied is the formulation of employee work shifts. In this example, information about a store's work schedule and employee information are provided, and the LLM 140 is asked to propose candidate employees to be assigned to each work time slot. Furthermore, the LLM 140 is used to determine a score representing the degree to which the employee assignment to each time slot matches the requirements of the employee and the store, and a work shift schedule is formulated that maximizes the total score and satisfies the constraints. While there are various methods for assigning employees to each time slot, the number of combination factors is reduced by using the LLM 140 to narrow down the candidates. Furthermore, because the scores for employee assignment are determined using the LLM 140, the burden on the user is reduced. In this way, appropriate work shifts can be formulated easily.

[0085] FIG. 11(A) shows an example of a prompt 1100 that the candidate element selection unit 111 sends to the LLM 140 when narrowing down the elements. The prompt 1100 includes a system prompt 1101 and a user prompt 1102. The system prompt 1101 includes task information that describes the content of the task to be performed by the LLM 140. This task information describes the presentation of pairs of time slots and employees, i.e., a list of employees to be assigned to each time slot, and the reason for recommendation. The user prompt 1102 includes information such as store working hours, and Contains information about the hours each employee is available to work.

[0086] 11(B) is a diagram showing an example of a response 1103 from the LLM 140 to the prompt 1100. The response 1103 includes candidate employee assignments for each time slot as specified in the prompt 1100 and reasons for recommending them. The candidate element selection unit 111 can parse the response 1103 to obtain the recommended employee assignments and the reasons for the recommendations.

[0087] The candidate element selection unit 111 presents the response results of the LLM 140 to the user, accepts the user's selection, and stores the selected allocation.

[0088] FIG. 12(A) shows an example of a prompt 1200 for obtaining a match score for each employee assignment. The prompt 1200 includes a system prompt 1201 and a user prompt 1202. The system prompt 1201 includes task information that describes the content of the task to be performed by the LLM 140. This task information describes that a score should be output for the assignment of employees to each time slot, taking into consideration the user (employer) preferences and employee preferences. The user prompt 1202 includes the user's preferences for work shifts and the preferences of each employee.

[0089] 12(B) is a diagram showing an example of a response 1203 from the LLM 140 to the prompt 1200. The response 1203 includes a score that indicates the degree of conformance with the user's and employee's preferences for the assignment of employees to each time slot, as specified in the prompt 1200.

[0090] The problem formulation unit 110 formulates the work shift planning problem based on the information obtained from the LLM 140 in this way. Specifically, the problem is formulated as follows: the work shift allocation selected by the user is included in the candidate elements, and a meal menu is selected from the candidate elements so that the score representing the user's satisfaction with each meal menu (degree of match with preferences) is maximized within a range that satisfies the constraints. The constraints are that at least one employee is assigned to each time slot, and the total working hours of each employee is within a predetermined time. In this example, the problem may be formulated without using the LLM 140 to determine the constraints.

[0091] In this way, work shifts can be planned taking into account the wishes of the user (employer) and employees. In this example, the candidate elements are narrowed down while appropriately reflecting the user's preferences, so that an appropriate solution can be provided to the user in a short processing time.

[0092] (Example 3: Creating a playlist) Another example of an optimization or planning problem to which the present disclosure can be applied is creating a music playlist. In this example, information about a user's music preferences is provided to the LLM 140, which then suggests song candidates that match the user's preferences. The LLM 140 is also asked to calculate the similarity between the suggested songs and one song selected by the user (a selected song). The solver 120 then sorts the songs based on the similarity to the selected song and the features of the other songs to create a playlist. The similarity to the selected song is calculated by the LLM 140, while the other features are obtained from information sources other than the LLM 140. The solver 120 creates the playlist by solving a Hamiltonian path minimization problem.

[0093] 13 shows a user interface 1300 in this example. The request query input area 1310 includes an input area 1311 for inputting the user's preferences for the playlist in text form, a record button 1312 for recording ambient audio, and an input area for inputting the total duration of the playlist. Both the preferences in text form and the recorded audio can be input. When the user presses the request send button 1320, the candidate element selection unit 111 acquires the information in the request query input area 1310 as the user's preferences (S401 in FIG. 4), and queries the LLM 140 to present songs that match the user's preferences (S402).

[0094] FIG. 14A shows an example of a prompt that the candidate element selection unit 111 sends to the LLM 140 to narrow down the songs that match the user's preferences. The prompt includes a system prompt 1401 and a user prompt 1402. The system prompt 1401 includes task information that describes the task that the LLM 140 performs. This task information describes the presentation of song candidates that match the user's preferences in order of similarity, as well as output rules. In this example, it specifies that songs that match the preferences in text format and songs similar to recorded audio are suggested. The user prompt 1402 includes text expressing the preferences entered in the input area 1311 and recorded audio data. FIG. 14B shows an example of a response 1403 from the LLM 140.

[0095] The candidate element selection unit 111 displays the contents of the response 1403 from the LLM 140 in the song candidate list 1330. Here, the candidate element selection unit 111 displays songs in descending order of match, and excludes subsequent songs if the accumulated playback time exceeds a specified time. Note that in case the user is not satisfied with the proposed song list, the user may be able to modify the query input and make a new inquiry.

[0096] When song candidates are presented by the LLM 140, the user inputs the first song in the playlist into the input area 1340. It is preferable that the song selected by the user in the song candidate list 1330 is copied to the input area 1340. In this example, all of the presented candidate list are considered candidates for inclusion in the playlist, so the user is not required to select a song from the candidate list, but the user may be allowed to further narrow down the candidates.

[0097] When the user presses the playlist creation button 1350, the objective function setting unit 112 requests the LLM 140 to sort the song candidate list 1330 based on the similarity to the first song. Figure 14(C) shows a prompt 1404 for sorting. Here, the prompt calculates the similarity based on the song title and lyrics, but similarity may also be calculated based on other information. Figure 14(D) shows an example of the LLM 140's response to this prompt.

[0098] Once a list of song candidates sorted by similarity to the first song is obtained, the problem formulation unit 110 formulates a Hamiltonian path minimization problem taking into account similarity to the first song and indices such as the tempo and energy of the songs, and solves this problem using the solver 120. Song indices can be obtained, for example, from a song information server. The Hamiltonian path minimization problem is a problem of determining the adoption cost of a path that passes through all given elements (songs). For example, each song can be represented as a point in a three-dimensional feature space using its similarity to the first song, tempo, and energy, and the problem of finding the shortest Hamiltonian path to travel through these points is solved. Unlike the salesman problem, this problem does not require returning to the starting point at the end.

[0099] The song list rearranged using the solver 120 is finally output to the playlist output area 1360. Although not shown in the figure, it is expected that this playlist will be used by exporting it to a music playback app, for example.

[0100] According to this example, it is possible to narrow down the candidates for songs to compose a playlist taking into consideration the user's preferences, and further to create a playlist that has small differences in similarity and other song attributes so that the transitions between songs are natural.

[0101] (Example 4: Recipe suggestions) Another example of an optimization problem or planning problem to which the present disclosure can be applied is proposing a cooking recipe. In this example, information about a user's cooking preferences is provided to the LLM 140, which then proposes several candidate dishes that match the user's preferences. Next, the user selects a candidate dish and inputs the desired calorie intake. Based on this input, the LLM 140 outputs information about the selected recipe, such as the ingredients, the calories of each ingredient, and the priority of the ingredients in the dish. If the user determines that changes are necessary, for example, the ingredients used are modified by making a new inquiry. When changes are no longer necessary, the obtained information is used to solve the optimization problem. The optimization problem is a knapsack problem in which ingredients must be selected so that a specified calorie limit is met while using high-priority ingredients.

[0102] 15A and 15B show a user interface 1500 in this example. A request query input area 1510 includes an input area 1512 for inputting the user's preference for food in text form, and an image input area 1511 for uploading an image of the inside of the refrigerator. When the user presses a request send button 1520, the candidate element selection unit 111 obtains the information in the request query input area 1510 as the user's preferences (S401 in FIG. 4) and queries the LLM 140 to present dishes that match the user's preferences.

[0103] FIG. 16A shows an example of a prompt that the candidate element selection unit 111 sends to the LLM 140 to output dishes that match the user's preferences. The prompt includes a system prompt 1601 and a user prompt 1602. The system prompt 1601 includes task information that describes the task that the LLM 140 is to perform. This task information describes the presentation of multiple dishes that match the user's preferences and the output rules. Here, it specifies that dishes that match the user's requirements are to be recommended along with the calorie content of one serving and the reason for recommendation. In addition, when suggesting dishes, ingredients present in the refrigerator may be taken into consideration, or dishes may be suggested based solely on the user's preferences. The user prompt 1602 includes text expressing the preferences entered in the input area 1512 and an image of a refrigerator 1511. FIG. 16B shows an example of a response 1603 from the LLM 140. The candidate element selection unit 111 displays the information obtained from the LLM 140 on the dish candidate presentation unit 1530 based on the response 1603 .

[0104] When the LLM 140 presents candidate dishes, the user inputs the dish they actually want to eat in the text input area 1541 of the input area 1540. At this time, the user can not only simply select the presented dish, but also input their preferences for the dish. In the example shown in the figure, a request for meat in a tofu hamburger steak has been input. The user also inputs a target calorie amount in the calorie input area 1540.

[0105] When the user presses the ingredient suggestion button 1550, the candidate element selection unit 111 requests the LLM 140 to suggest candidate ingredients for the dish. Based on the response from the LLM 140, the candidate ingredients are displayed in the candidate ingredient list 1560 (FIG. 15B). FIG. 16(C) shows an example of a prompt that the candidate element selection unit 111 sends to the LLM 140 to output ingredients that match the user's preferences. The prompt includes a system prompt 1604 and a user prompt 1605. The system prompt 1604 includes task information describing the task that the LLM 140 is to perform. This task information describes the suggestion of candidate ingredients for the dish that match the user's preferences and calorie restrictions, as well as output rules. Here, we request that the priority of ingredients used in the dish (in this example, two levels, required and optional, are used, but three or more levels may be used) and the required amount and calories for each ingredient be output. Furthermore, when suggesting candidate ingredients, the system also requests reference to an image of the refrigerator. The user prompt 1605 displays the input data entered in the input area 1541. Text expressing the user's preferences, the total calorie amount entered in input area 1542, and an image of a refrigerator 1511 are added. Fig. 16(D) shows an example of a response 1606 from the LLM 140. Based on the response 1606, the candidate element selection unit 111 displays information obtained from the LLM 140 in the ingredient candidate presentation area 156.

[0106] If the user determines that adjustments are necessary after viewing the ingredient candidates, they can change the input content in the input area 1541 and press the ingredient suggestion button 1550 again to re-suggest the ingredients. When re-entering, requests may be made, for example, to remove a suggested ingredient (e.g., "Do not use ingredient A"), to change a suggested ingredient to another ingredient (e.g., "Please use ingredient B instead of ingredient A"), or to change the priority of an ingredient (e.g., "Make sure to include ingredient A"). The LLM 140 will change the ingredient candidates in accordance with the new request. The user repeats this process until they are satisfied with the ingredient candidates, and if they determine that no adjustments are necessary, they press the recipe suggestion button 1570.

[0107] When the recipe suggestion button 1570 is pressed, the problem formulation unit 110 formulates a combinatorial optimization problem for ingredients. The candidate combinations are the candidate ingredients proposed by the LLM 140. A knapsack problem is formulated to select, from these candidate elements, the ingredient candidate that will achieve the highest priority-based score, with the constraint that the total calorie content must be within an upper limit. A constraint that required ingredients must be used may also be added. The knapsack problem formulated in this way is solved using the solver 120 to determine the ingredients to be used in the dish. The plan creation device 100 then provides the determined ingredients to the LLM 140, causing it to output the steps for creating the dish, and outputs the ingredients to be used and the cooking steps to the recipe display unit 1580.

[0108] According to this example, it is possible to narrow down the dishes that suit the user's preferences and the candidate ingredients to be used for those dishes, and a combinatorial optimization problem is solved using the narrowed-down ingredients, making it possible to efficiently propose cooking recipes that suit the user's preferences.

[0109] In this example, the ingredient candidate list 1560 is displayed and the solution to the optimization problem and the procedure are presented after the user's confirmation; however, the process may proceed without presenting the ingredient candidate list 1560, and the recipe may be displayed, and the user may request changes after viewing the resulting recipe.

[0110] <Variation 2> This modification describes a method for adapting to individual users when narrowing down candidate elements using an LLM. In this modification, multiple types of candidate element suggestion systems are prepared. The multiple suggestion systems may be realized using different LLMs, or may be realized by providing different prompts to the same LLM to assign different characteristics. To assign different characteristics to an LLM, specifically, multiple system prompts may be prepared and queries may be made using each prompt. For example, system prompt 501 in Figure 5 assigns the characteristic "You are a travel agent who proposes attractive travel plans to users." However, the LLM's characterization can be changed by changing this wording.

[0111] The candidate element selection unit 111 provides prompts to multiple candidate narrowing systems (i.e., different LLMs or differently characterized LLMs) and acquires candidate elements from each of them. For example, if there are N candidate narrowing systems, the candidate element selection unit 111 assigns a probability Pn (where 0≦Pn≦1, ΣPn=1) to the nth system. If the number of candidates proposed to the user is M, the probability that the proposal of the nth system will be adopted for the mth candidate is set to Pn. The probability Pn assigned to each system is determined by the probability of the system being selected by the user. It is advisable to learn the selection probability Pn of each system so as to maximize the number of candidate elements selected by the user. For example, the candidate element selection unit 111 uses a linear bandit algorithm to learn the selection probability Pn of each system so as to maximize the reward, with the number of candidates selected by the user being the reward. This makes it possible to more accurately narrow down the candidates that match the user's preferences.

[0112] <Variation 3> The information provided as the user prompt is not limited to text data, but may include at least one of image data, video data, audio data, and sensor data (time-series data of sensor values). The sensor information may be, for example, at least one of an electroencephalogram sensor, a speed sensor, an acceleration sensor, a gyro sensor, an ultrasonic sensor, and a radiation sensor.

[0113] <Variation 4> In the above-described embodiment, text entered by the user is sent as a user prompt to the LLM 140. Therefore, if an inappropriate input is made, it is conceivable that malfunction or inappropriate output will occur. Therefore, before sending the user prompt to the LLM, the LLM 140 may be made to determine whether the user prompt contains inappropriate content, and if it is determined to be inappropriate, processing may be terminated.

[0114] <Other embodiments> The above-described embodiment is merely an example, and the present disclosure can be modified and implemented as appropriate within the scope that does not deviate from the gist of the disclosure.

[0115] <Additional Notes> An information processing device that determines an element that satisfies a constraint condition and optimizes an objective function from among a plurality of elements, a problem setting unit (110, 111) that selects candidate elements from the plurality of elements based on an output (503) obtained by providing an input (502) from a user to a large-scale language model (140); a solver (120) that selects the candidate elements and determines an element that satisfies the constraints and optimizes the objective function; An information processing device (100) comprising: [Explanation of symbols]

[0116] 100: Information processing device 110: Problem formulation unit 111: Candidate element selection unit 112: Objective function setting unit 113: Constraint condition setting unit 120: Solver (solving unit) 130: Result output unit 140: LLM

Claims

1. An information processing device that determines an element that satisfies a constraint condition and optimizes an objective function from among a plurality of elements, a problem setting unit that selects candidate elements from the plurality of elements based on an output obtained by providing an input from a user to a large-scale language model; a solver that selects the candidate elements to determine an element that satisfies the constraints and optimizes the objective function; An information processing device comprising:

2. The input from the user is information expressing the user's preferences in natural language.

2. The information processing apparatus according to claim 1, wherein:

3. The input from the user further includes at least one of image data, video data, audio data, and sensor data.

3. The information processing apparatus according to claim 2, wherein:

4. the problem setting unit presents tentative candidate elements selected from the plurality of elements to the user based on an output obtained by providing an input from the user to a large-scale language model, and determines an element selected by the user from the tentative candidate elements as the candidate element; 2. The information processing apparatus according to claim 1, wherein:

5. the problem setting unit retrieves text representing an explanation of the tentative candidate element from the large-scale language model and presents the text to the user; 5. The information processing apparatus according to claim 4,

6. the objective function is a sum of scores assigned to each of the plurality of elements; the problem setting unit determines a score to be assigned to the candidate element based on an output obtained by providing the input from the user to the large-scale language model; 2. The information processing apparatus according to claim 1, wherein:

7. the problem setting unit instructs the large-scale language model to return a degree of match between the user's preference included in the input from the user and the candidate element, and determines the degree of match obtained from the large-scale language model as the score of the candidate element.

7. The information processing apparatus according to claim 6,

8. the constraint condition includes a condition that a sum of costs assigned to each of the plurality of elements is equal to or less than a threshold; the problem definition unit determines a cost to be assigned to the candidate element based on an output obtained by providing an input from the user to the large-scale language model; 2. The information processing apparatus according to claim 1, wherein:

9. the optimization is a combinatorial optimization problem, The solving unit determines a combination of elements so as to optimize the sum of scores assigned to the selected elements while satisfying a constraint that the cost assigned to the selected elements is equal to or less than a threshold.

2. The information processing apparatus according to claim 1, wherein:

10. The problem setting unit The input from the user is further processed by a plurality of large-scale language models or a plurality of different characterizations. selecting tentative candidate elements to be presented to the user from among tentative candidate elements obtained by applying the tentative candidate elements to the large-scale language model; determining an element selected by the user from the tentative candidate elements presented to the user as the candidate element; 2. The information processing apparatus according to claim 1, wherein:

11. the problem setting unit learns a selection probability when selecting tentative candidate elements to be presented to a user from tentative candidate elements obtained from the plurality of large-scale language models or the plurality of large-scale language models with different characteristics, so as to maximize the number of tentative candidate elements selected by the user.

9. The information processing apparatus according to claim 8,

12. An information processing method performed by an information processing device, which determines an element from a plurality of elements that satisfies a constraint condition and optimizes an objective function, a problem setting unit step of selecting candidate elements from the plurality of elements based on an output obtained by providing an input from a user to a large-scale language model; a solution step of determining an element that satisfies the constraints and optimizes the objective function for the candidate element; An information processing method comprising:

13. A program for causing a computer to function as each of the means of the information processing device according to any one of claims 1 to 10.

14. A program for causing a computer to execute each step of the information processing method according to claim 12.

Images