Information processing apparatus, inference model generation method, inference method, and inference program

Abstract

To make it possible to achieve improvement in inference of a solution to a combinational optimization problem.SOLUTION: An information processing apparatus includes: a data acquisition unit that acquires input data pertaining to a combinational optimization problem for which a solution is to be obtained; and an inference unit that infers a solution in accordance with the input data using an inference model which has been generated by reinforcement learning for inferring a solution to the optimization problem, the inference model including an encoder that extracts a feature amount of the input data and a decoder that generates information indicating a solution to the optimization problem using the feature amount extracted by the encoder.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, a method for generating an inference model, an inference method, and an inference program. [Background technology]

[0002] There are known techniques for solving business issues by applying various combinatorial optimization problems, such as delivery planning problems, to business, etc. For example, Patent Document 1 listed below discloses a transportation plan creation support method that supports the creation of an optimal transportation plan in a transportation plan in which multiple delivery vehicles perform their delivery tasks while satisfying various constraints. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2024-17959 Summary of the Invention [Problem to be solved by the invention]

[0004] The transportation plan creation support method described in Patent Document 1 has room for improvement in that the input data that must be input to create an optimal transportation plan and the output data that is output by the optimization calculation are both fixed. This is not limited to transportation plans, but is common when dealing with any combinatorial optimization problem.

[0005] The present disclosure has been made in view of the above problems, and an exemplary purpose thereof is to provide a technique that enables improvements in inferring solutions to combinatorial optimization problems. [Means for solving the problem]

[0006] An information processing device according to an exemplary aspect of the present disclosure includes: data acquisition means for acquiring input data related to a combinatorial optimization problem for which a solution is sought; and inference means for inferring a solution corresponding to the input data using an inference model for inferring a solution to the optimization problem, the inference model being generated by reinforcement learning, the inference model including an encoder for extracting features of the input data and a decoder for generating information indicating the solution to the optimization problem using the features extracted by the encoder.

[0007] A method for generating an inference model according to an exemplary aspect of the present disclosure includes an inference process in which at least one processor uses an inference model that infers a solution to a combinatorial optimization problem, the inference model including an encoder that extracts features of input data input to the inference model and a decoder that generates information indicating a solution to the optimization problem from the features extracted by the encoder, to infer a solution based on the input data; and an update process in which the inference model is updated by reinforcement learning based on evaluation results of the solution obtained in the inference process.

[0008] An inference method according to an exemplary aspect of the present disclosure includes: a data acquisition process in which at least one processor acquires input data related to a combinatorial optimization problem for which a solution is sought; and an inference process in which an inference model for inferring a solution to the optimization problem, the inference model being generated by reinforcement learning, the inference model including an encoder that extracts features of the input data and a decoder that generates information indicating a solution to the optimization problem using the features extracted by the encoder, to infer a solution corresponding to the input data.

[0009] An inference program according to an exemplary aspect of the present disclosure causes a computer to function as inference means for inferring a solution based on input data, using data acquisition means for acquiring input data related to a combinatorial optimization problem for which a solution is sought, and an inference model for inferring a solution to the optimization problem, the inference model being generated by reinforcement learning and including an encoder for extracting features of the input data and a decoder for generating information indicating a solution to the optimization problem using the features extracted by the encoder. [Effects of the Invention]

[0010] According to an exemplary aspect of the present disclosure, an exemplary effect can be achieved in that a technique can be provided that enables improvements in inferring solutions to combinatorial optimization problems. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram illustrating a configuration of an information processing device according to the present disclosure. [Figure 2] FIG. 1 is a flow diagram illustrating the flow of an inference method according to the present disclosure. [Figure 3] FIG. 10 is a block diagram showing a configuration of another information processing device according to the present disclosure. [Figure 4] A flow chart showing the flow of the inference model generation method related to the present disclosure. [Figure 5] 1 is a diagram illustrating an overview of an optimization support system according to the present disclosure. [Figure 6] FIG. 10 is a block diagram showing the configuration of yet another information processing device according to the present disclosure. [Figure 7] A diagram showing an example configuration of an encoder included in the inference model. [Figure 8] FIG. 8 is a diagram showing an example of inferring an optimal delivery route using an inference model including the encoder shown in FIG. 7. [Figure 9] A diagram showing an example of incremental learning of an inference model. [Figure 10] 7 is a flowchart showing the flow of processing executed by the information processing device shown in FIG. 6. [Figure 11] FIG. 1 is a block diagram illustrating a configuration of a computer that functions as an information processing device according to the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0012] The following are examples of embodiments of the present invention. However, the present invention is not limited to the exemplary embodiments shown below, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technologies (part or all of the products or methods) employed in the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, embodiments obtained by appropriately omitting some of the technologies employed in the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, the effects mentioned in the exemplary embodiments shown below are examples of effects expected in the exemplary embodiments, and do not define the scope of the present invention. In other words, embodiments that do not exhibit the effects mentioned in the exemplary embodiments shown below may also be included in the scope of the present invention.

[0013] First Exemplary Embodiment A first exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. This exemplary embodiment is a basic form for each of the exemplary embodiments described below. Note that the scope of application of each technique employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technique employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise. Furthermore, each technique shown in the drawings referenced to explain this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise.

[0014] (Configuration of information processing device 1) The configuration of the information processing device 1 will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the information processing device 1. As shown in Fig. 1, the information processing device 1 includes a data acquisition unit 11 and an inference unit 12.

[0015] The data acquisition unit 11 acquires input data related to a combinatorial optimization problem (hereinafter referred to as "optimization problem") for which a solution is to be sought. This input data may be any data containing information necessary to infer a solution to the optimization problem, and various types of data such as text, numerical values, and images may be used.

[0016] The data used in real-world optimization problems takes different forms depending on the characteristics and requirements of the problem. As described above, the information processing device 1 can accept various types of data as input data, and is therefore adaptable to the data characteristics of real-world optimization problems.

[0017] For example, if the optimization problem is the traveling salesman problem, the input data may be a list indicating the coordinates of each target point. These coordinates are usually expressed as latitude and longitude or as points on an XY plane. Alternatively, the input data may be information indicating each target point and the routes connecting those points (for example, graph data in which each point is represented as a node and the routes connecting those points are represented as edges).

[0018] Furthermore, the input data may include constraints for the optimization problem, or various information that is not essential for solving the optimization problem but is preferable to consider. For example, in the traveling salesman problem, suppose we want to optimize the same route assuming asymmetry, such as the cost of travelling there being different on the way there and back. In this case, we can include a cost matrix representing this asymmetry in the input data.

[0019] Furthermore, if the optimization problem is a delivery planning problem, the input data can include information indicating constraints such as the location information of delivery destinations and the like, as well as information on each mobile vehicle (e.g., delivery truck) used for delivery and its loading capacity. If the optimization problem is a pick-and-delivery problem, the input data can include information on the receiving and handing destinations. If the optimization problem is a job-shop scheduling problem, the total work time must be minimized, taking into account that work time changes depending on the order in which tasks are performed. By including a cost matrix in the input data, it is possible to determine the optimal execution order, taking into account such changes in work time.

[0020] The information processing device 1 is suitable for inferring solutions to optimization problems that require consideration of order, such as those described above, but can also be applied to inferring solutions to problems that do not require consideration of order, such as the knapsack problem. When the optimization problem is the knapsack problem, the input data is a list of items, and each item is represented by a pair of weight and value (price, etc.). The input data may be converted into a predetermined format depending on the type of problem.

[0021] The inference unit 12 uses an inference model for inferring a solution to the optimization problem to infer a solution corresponding to the input data acquired by the data acquisition unit 11. This inference model is an inference model generated by reinforcement learning, and includes an encoder that extracts features of the input data, and a decoder that generates information indicating a solution to the optimization problem using the features extracted by the encoder.

[0022] As described above, the information processing device 1 includes a data acquisition unit 11 that acquires input data related to a combinatorial optimization problem for which a solution is to be found, and an inference unit 12 that infers a solution based on the input data using an inference model for inferring a solution to the optimization problem, the inference model being generated by reinforcement learning and including an encoder that extracts features of the input data and a decoder that generates information indicating the solution to the optimization problem using the features extracted by the encoder. This configuration has the effect of enabling improvements in inferring solutions to combinatorial optimization problems.

[0023] More specifically, according to the above configuration, feature extraction is performed by the encoder, and the extracted feature is used to generate information indicating a solution to the optimization problem in the decoder. In other words, each data point included in the input data is converted by the encoder into a high-dimensional internal state vector (also called an "embedding vector"). This conversion allows the relationships and structure between the data to be expressed by a model. In this way, since the input data is not directly used to generate information indicating a solution to the optimization problem, various input data such as those described above can be accepted as long as the encoder can extract features from it. This allows the information processing device 1 to have a high degree of input flexibility.

[0024] Furthermore, with the above configuration, the decoder generates information indicating a solution to the optimization problem. That is, the decoder generates an optimal solution using the internal state vector obtained from the encoder. Since past inference results are reflected in the learning of the inference model, the optimal solution generated in this way can be said to have been determined taking into consideration the internal state vector generated by the encoder, past inference results, and their evaluation results.

[0025] The output format of the decoder can be determined as appropriate when designing the inference model, and the output format can also be changed through additional learning. For this reason, the information processing device 1 has a high degree of freedom in output.

[0026] However, if various input data can be applied, it is possible that the input data obtained may be incomplete in inferring a solution to the optimization problem. For example, when solving an optimization problem that deals with geographical information, using only words or simple labels indicating geographical locations as input data may not fully capture important elements such as actual distance, direction, and locational relationships. For example, it is difficult to directly read the actual distance, direction, and transportation accessibility from the text "I want to go from New York to Tokyo."

[0027] However, the inference model used in the information processing device 1 is generated by reinforcement learning. Through the reinforcement learning process, the encoder is updated so that it can extract features that more accurately reflect, for example, geographical structures and other important relationships that are not directly expressed in the input data. Therefore, the information processing device 1 can obtain appropriate inference results even when incomplete input data is input.

[0028] Furthermore, combinatorial optimization problems such as the traveling salesman problem frequently occur in real business, but are known to be NP-hard problems. When solving such optimization problems using an optimization-based method such as that described in Patent Document 1, it is known that the time required to calculate an optimal solution increases significantly as the problem size increases, making it difficult to obtain an optimal solution within a practical time frame. In this regard, the information processing device 1 performs inference using a pre-trained inference model, so that the increase in the time required for inference is suppressed even if the problem size increases, making it practical to apply the method to various businesses.

[0029] In this way, the information processing device 1 can collect information from various input sources and generate an appropriate solution to a combinatorial optimization problem. The encoder converts the input data into a high-dimensional feature space to calculate the features of the data, allowing a subsequent decoder to calculate an optimal solution based on these features. Therefore, the information processing device 1 can incorporate information necessary to derive an optimal solution, such as accurate geographical relationships, into its internal state (the features output by the encoder), regardless of the form of the input data.

[0030] (inference program) The functions of the information processing device 1 described above can also be realized by a program. The inference program according to this exemplary embodiment causes a computer to function as inference means for inferring a solution based on input data using an inference model that includes: data acquisition means for acquiring input data related to a combinatorial optimization problem for which a solution is to be found; and an inference model for inferring a solution to the optimization problem, generated by reinforcement learning, the inference model including an encoder that extracts features from the input data and a decoder that generates information indicating the solution to the optimization problem using the features extracted by the encoder. Therefore, the inference program according to this exemplary embodiment has the effect of enabling improvements in inferring solutions to combinatorial optimization problems.

[0031] (Flow of reasoning method) The flow of the inference method will be described with reference to Fig. 2. Fig. 2 is a flow diagram showing the flow of the inference method. Note that the execution entity of each step in this inference method may be a processor provided in the information processing device 1, or may be a processor provided in another device. In other words, the execution entities of each step may be processors provided in different devices.

[0032] In S11 (data acquisition process), at least one processor acquires input data related to a combinatorial optimization problem for which a solution is to be obtained.

[0033] In S12 (inference processing), at least one processor infers a solution corresponding to the input data using an inference model for inferring a solution to the optimization problem, the inference model being generated by reinforcement learning and including an encoder that extracts features of the input data and a decoder that generates information indicating a solution to the optimization problem using the features extracted by the encoder.

[0034] As described above, the inference method according to this exemplary embodiment includes a data acquisition process performed by at least one processor to acquire input data related to a combinatorial optimization problem for which a solution is to be found, and an inference process to infer a solution based on the input data using an inference model for inferring a solution to the optimization problem, the inference model having been generated by reinforcement learning and including an encoder that extracts features from the input data and a decoder that generates information indicating the solution to the optimization problem using the features extracted by the encoder. This advantageously enables improvements in inferring solutions to combinatorial optimization problems.

[0035] (Configuration of information processing device 2) The configuration of the information processing device 2 will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the configuration of the information processing device 2. As shown in Fig. 3, the information processing device 2 includes an inference unit 21 and a learning unit 22.

[0036] The inference unit 21 infers a solution corresponding to the input data using an inference model that infers a solution to a combinatorial optimization problem and includes an encoder that extracts features from input data input to the inference model and a decoder that generates information indicating a solution to the optimization problem from the features extracted by the encoder. The input data used for this inference, like the input data used for inference in the information processing device 1 described above, may be any data that includes information necessary to infer a solution to the optimization problem and from which features can be extracted by the encoder.

[0037] The learning unit 22 updates the inference model by reinforcement learning based on the evaluation result of the solution inferred by the inference unit 21. Any algorithm for reinforcement learning may be used. For example, the learning unit 22 may update the inference model (more precisely, update each parameter of the encoder and decoder included in the inference model) by a method such as a policy gradient method.

[0038] As described above, the information processing device 2 includes an inference unit 21 that infers a solution according to the input data using an inference model that infers a solution to a combinatorial optimization problem, the inference model including an encoder that extracts features of input data input to the inference model and a decoder that generates information indicating a solution to the optimization problem from the features extracted by the encoder, and a learning unit 22 that updates the inference model by reinforcement learning based on the evaluation results of the solution inferred by the inference unit 21.

[0039] The above configuration makes it possible to generate an inference model that can accept various input data. Furthermore, the output format of the decoder included in this inference model can be determined appropriately when the inference model is designed, and the output format can also be changed through fine tuning. Therefore, it can be said that the output of the inference model generated by the information processing device 2 has a high degree of freedom. Therefore, the above configuration has the effect of enabling improvements in the inference of solutions to combinatorial optimization problems.

[0040] During reinforcement learning, the solution inferred by the inference unit 21 is evaluated. The solution is evaluated based on a reward calculated using a predetermined reward function. This reward can also be described as an evaluation value indicating how well the inferred solution satisfies the actual requirements and objectives of the target optimization problem. In reinforcement learning performed by the learning unit 22, the type of reward function to be applied may be determined appropriately depending on the target optimization problem, the constraints to be applied, and the like. For example, the reward for an optimization problem dealing with geographical information may be calculated based on actual data such as a map. As a specific example, in the traveling salesman problem, a higher reward may be given to a route with a shorter distance on a map.

[0041] (Study Program) The functions of the information processing device 1 described above can also be realized by a program. The inference program according to this exemplary embodiment causes a computer to function as an inference means for inferring a solution corresponding to input data using an inference model that infers a solution to a combinatorial optimization problem, the inference model including an encoder that extracts features from input data input to the inference model and a decoder that generates information indicating a solution to the optimization problem using the features extracted by the encoder, and as a learning means for updating the inference model through reinforcement learning based on evaluation results of the solution inferred by the inference means. Therefore, the inference program according to this exemplary embodiment has the effect of enabling improvements in inferring solutions to combinatorial optimization problems.

[0042] (Flow of inference model generation method) The flow of the method for generating an inference model will be explained with reference to Figure 4. Figure 4 is a flow diagram showing the flow of the method for generating an inference model. Note that the entity that executes each step in this method for generating an inference model may be a processor provided in the information processing device 2, or may be a processor provided in another device. In other words, the entity that executes each step may be a processor provided in each different device.

[0043] In S21 (inference processing), at least one processor infers a solution corresponding to the input data using an inference model that infers a solution to a combinatorial optimization problem, the inference model including an encoder that extracts features of input data input to the inference model, and a decoder that generates information indicating a solution to the optimization problem from the features extracted by the encoder.

[0044] In S22 (update processing), at least one processor updates the inference model by reinforcement learning based on the evaluation result of the solution inferred in S21.

[0045] As described above, the method for generating an inference model according to this exemplary embodiment includes an inference process in which at least one processor uses an inference model that infers a solution to a combinatorial optimization problem, the inference model including an encoder that extracts features from input data input to the inference model and a decoder that generates information indicating a solution to the optimization problem using the features extracted by the encoder, to infer a solution based on the input data; and an update process in which the inference model is updated by reinforcement learning based on evaluation results of the solution obtained in the inference process. This provides the advantage of enabling improvements in inferring solutions to combinatorial optimization problems. Note that the inference model generated by the above generation method is also within the scope of the present invention.

[0046] Second Exemplary Embodiment A second exemplary embodiment, which is one example of an embodiment of the present invention, will be described in detail with reference to the drawings. Components having the same functions as those described in the above exemplary embodiment will be assigned the same reference numerals, and their description will be omitted as appropriate. The scope of application of each technology employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technology employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical hindrance occurs. Furthermore, each technology shown in each drawing referenced to explain this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical hindrance occurs.

[0047] (Configuration of Optimization Support System 5) The configuration of the optimization support system 5 will be described with reference to Fig. 5. Fig. 5 is a diagram showing an overview of the optimization support system 5. The optimization support system 5 is a system equipped with a function for inferring solutions to combinatorial optimization problems. As shown in the figure, the optimization support system 5 includes an information processing device 3 that performs various processes for realizing the above functions, and a terminal device 4 used by a user of the optimization support system 5.

[0048] An example of inferring an optimal delivery route for a package using the optimization support system 5 will be described below. When a single vehicle (e.g., a truck) is used for delivery, the optimization problem to be solved becomes a traveling salesman problem, and when multiple vehicles are used, the optimization problem to be solved becomes a delivery planning problem. The optimization support system 5 is capable of inferring a solution to any combinatorial optimization problem. For example, the optimization support system 5 can also infer a solution to the knapsack problem.

[0049] A user of the optimization support system 5 is, for example, a delivery planner at a logistics company. Every day, the user processes orders with a large number of pickup and delivery destinations and must plan efficient delivery routes using a limited number of trucks and drivers. First, the user inputs various pieces of information required to infer the optimal delivery route for a package via a GUI (Graphical User Interface) displayed on the terminal device 4.

[0050] 5 shows an example in which the terminal device 4 is a smartphone. The terminal device 4 may have the functions of accepting input of necessary information via a GUI, transmitting the input information to the information processing device 3 via a network such as the Internet, receiving the result of inference based on the information from the information processing device 3, and presenting the result to the user. For example, the terminal device 4 may be a personal computer.

[0051] In the example of Fig. 5, a form for inputting the pickup destination, desired pickup time slot, delivery destination, desired delivery time slot, and information about the package (such as the weight and size of the package) is displayed on the display unit of the terminal device 4. The user inputs the above information into this form via the touch panel of the terminal device 4, and once the input is complete, the user simply selects a button labeled "Optimize." This causes the input information to be sent to the information processing device 3 via the network.

[0052] The information processing device 3 acquires the transmitted information as input data for an optimization problem that infers the optimal delivery route, and starts the process of optimizing the delivery route. Specifically, the information processing device 3 first extracts feature quantities from the input data using a trained encoder. This process can also be described as converting the input data into an internal state vector. The internal state vector (feature quantities) includes information such as the geographical positional relationship between the pickup destination and delivery destination, time constraints due to the desired time slot, and load capacity constraints due to the weight and size of the package.

[0053] Next, the information processing device 3 optimizes the internal state vector while taking into account past delivery performance data. Using this optimized internal state vector, the trained decoder generates the most efficient delivery route that satisfies the constraints of the number of trucks and driver working hours.

[0054] The generated optimal delivery route is transmitted to the terminal device 4 via the network and displayed on the display unit of the terminal device 4. The delivery route may be presented in any manner. For example, the information processing device 3 may display the generated delivery route superimposed on a map image, as shown in FIG. 5. The information processing device 3 may also accept user modifications to the displayed delivery route. User modifications are made via an input device such as a touch panel provided on the terminal device 4.

[0055] Then, the user simply selects the button labeled "Confirm" with the desired delivery route displayed. This starts route guidance along the delivery route. The route guidance may be executed by the information processing device 3, or may be executed by a navigation app installed on the terminal device 4 or an in-vehicle navigation device. This allows the user to confirm the delivery route for which they are responsible and use the navigation function to efficiently carry out collection and delivery.

[0056] Furthermore, the progress of delivery is fed back to the information processing device 3 in real time via the GPS (Ground Positioning System) function of the terminal device 4. Based on this feedback, the information processing device 3 then distinguishably displays points where delivery has been completed from points where delivery is not yet completed. This allows the progress of delivery to be grasped at a glance. Furthermore, once delivery is completed, data such as the actual time taken for delivery and the distance traveled is sent to the information processing device 3 and is used to optimize the next delivery route.

[0057] As described above, Optimization Support System 5 automatically generates optimal delivery routes that take complex constraints into account, significantly reducing the time and effort required to create delivery plans and significantly streamlining users' daily work. Furthermore, continuous learning using past data improves the quality of delivery routes on a daily basis. Optimization Support System 5 has a significant effect on various aspects of logistics operations, such as reducing delivery costs, improving customer satisfaction, and reducing the burden on drivers. Optimization Support System 5 is expected to be a source of competitiveness in the logistics industry. Optimization Support System 5 can also be applied to the medical and healthcare fields, such as optimizing doctor visit routes and rehabilitation and training menus.

[0058] (Configuration of information processing device 3) The configuration of the information processing device 3 will be described with reference to Fig. 6. Fig. 6 is a block diagram showing the configuration of the information processing device 3. As shown in the figure, the information processing device 3 includes a control unit 30 that controls each unit of the information processing device 3, and a storage unit 31 that stores various data used by the information processing device 3. The information processing device 3 also includes a communication unit 32 that enables the information processing device 3 to communicate with other devices (e.g., the terminal device 4 shown in Fig. 5), an input unit 33 that accepts input to the information processing device 3, and an output unit 34 that enables the information processing device 3 to output data. The control unit 30 includes a data acquisition unit 301, an inference unit 302, a presentation unit 303, a correction unit 304, a guidance unit 305, and a learning unit 306. The storage unit 31 also stores an inference model 311.

[0059] The data acquisition unit 301 acquires input data related to a combinatorial optimization problem for which a solution is to be found. The input data acquired by the data acquisition unit 301 may contain information necessary to infer a solution to the optimization problem, similar to the input data described in the first exemplary embodiment, and may be data from which features can be extracted by an encoder included in the inference model 311.

[0060] For example, the data acquiring unit 301 may acquire the input data including at least one of text data describing the optimization problem in a natural language and image data related to the optimization problem. This provides the effect of enabling a user to easily input input data that will result in a desired optimal solution, in addition to the effect provided by the information processing device 1 of exemplary embodiment 1.

[0061] For example, the user may input text such as "The current location is point A, and I would like to make deliveries at points B, C, D, and F" as input data. The user may also input a marking image of a map image in which each point that needs to be passed is marked as input data. Even with data in this format, the encoder included in the inference model 311 can extract features, and the decoder can generate a solution to the optimization problem based on the features. Voice input by the user may also be accepted. In this case, the terminal device 4 or the information processing device 3 may convert the user's voice input into text data and use it as input data.

[0062] The inference unit 302 uses an inference model 311 for inferring a solution to the optimization problem to infer a solution corresponding to the input data acquired by the data acquisition unit 301. The inference model 311 includes an encoder that extracts features of the input data and a decoder that generates information indicating a solution to the optimization problem using the features extracted by the encoder, and is an inference model generated by reinforcement learning that applies a reward for evaluating the solution to the optimization problem. Note that the inference model 311 does not necessarily need to be stored in the information processing device 3, but may be stored in another device. When using an inference model 311 stored in another device, the inference unit 302 instructs the other device to perform inference using the inference model 311 and acquires the inference result from the other device.

[0063] The inference model 311 may be, for example, a model that is generated by learning about multiple target locations and the routes between the locations, and that infers an optimal route that passes through each of the target locations while satisfying predetermined conditions. Such inference is required in various businesses, including the above-mentioned package delivery. Therefore, the above configuration has the effect of contributing to the efficiency of such businesses. The problem of inferring an optimal route may be, for example, a traveling salesman problem or a delivery planning problem.

[0064] As described above, the inference unit 302 causes the inference model 311 decoder to generate information indicating the solution to the optimization problem. The output format of the decoder can be determined appropriately when the inference model is designed. For example, the decoder may be trained to generate at least one of image data indicating the solution to the optimization problem and text data indicating the solution to the optimization problem. This provides the effect of presenting the solution to the optimization problem to the user in a format that is intuitively easy to recognize, in addition to the effect provided by the information processing device 1 of exemplary embodiment 1.

[0065] For example, if the optimization problem to be solved involves inferring the optimal route that passes through multiple target locations, an inference model 311 may be used that includes a decoder trained to output image data in which the optimal route is superimposed on a map image, or text data providing guidance about the optimal route.

[0066] The inference model 311 may be any model including an encoder and a decoder, and may be, for example, a variational autoencoder or a Transformer model. In this exemplary embodiment, an example in which the inference model 311 is a Transformer model will be described.

[0067] The presentation unit 303 presents the inference result of the inference unit 302 to the user. The presentation manner of the inference result is arbitrary. For example, the presentation unit 303 may present the inference result by displaying an image showing the inference result on the terminal device 4, as in the example of FIG. 5. Furthermore, for example, the presentation unit 303 may present the inference result via the output unit 34. In this way, the presentation unit 303 may output the inference result to an output device (output unit 34) included in the information processing device 3, or may output the inference result to an output device external to the information processing device 3 (for example, the terminal device 4). Furthermore, the presentation unit 303 may output the inference result by voice or by printing.

[0068] The correction unit 304 accepts a user's correction to the inference result of the inference unit 302. For example, the correction unit 304 may accept the correction via the terminal device 4 as in the example of Fig. 5, or may accept the correction via the input unit 33. When the correction unit 304 accepts the correction, the presentation unit 303 reflects the correction in the presented inference result.

[0069] The guidance unit 305 provides route guidance so that the user can make delivery along the optimal delivery route inferred by the inference unit 302 (or the delivery route after the correction, if the correction unit 304 accepts the correction). The route guidance may be executed by the guidance unit 305 or may be executed by another device.

[0070] When the guidance unit 305 executes route guidance, the guidance unit 305 displays the user's current position on a map image, as well as the optimal delivery route inferred by the inference unit 302 and each point on the delivery route (such as the collection point and delivery destination point).The guidance unit 305 then guides the user by voice and image so that the user can move along the delivery route and collect and deliver the package.

[0071] On the other hand, when route guidance is to be performed by another device (for example, a terminal device 4 with a navigation app installed or an in-vehicle navigation device), the guidance unit 305 may notify the device of the delivery route and instruct the device to provide route guidance along the delivery route.

[0072] Similar to the learning unit 22 described in the first exemplary embodiment, the learning unit 306 updates the inference model 311 by reinforcement learning based on the evaluation results of the solution inferred by the inference unit 302. Reinforcement learning is performed within a framework in which a process of performing an action determined according to a policy in a certain state, calculating a reward for the action based on the state after the action, and repeating this process to update the policy so as to maximize the total reward. In reinforcement learning by the learning unit 306, the inference model 311 is applied as the "policy" in this framework. That is, the learning unit 306 repeats the process of having the inference unit 302 infer a next point after a certain point and calculating a reward for the inference result based on the state when the inference unit 302 has moved to the next point until the route is completed, and updates the inference model 311 so as to maximize the total reward when the completed route is applied (in other words, the evaluation of the entire inference result).

[0073] (Configuration of inference model 311) As mentioned above, the inference model 311 may be a Transformer model. Here, a Transformer model that has been trained on word sequences in natural language can infer the word that follows a target word, and by repeating this inference, it is possible to generate a sentence.

[0074] In contrast, when the inference model 311 is a Transformer model, instead of learning words and their order, it is sufficient to learn each point in the area for which the optimal delivery route is to be determined and the order in which to pass through each point. This makes it possible to generate an inference model 311 that generates an optimal delivery route by repeating the process of inferring the next point to be headed from a specified point. The same applies to solving other combinatorial optimization problems such as the knapsack problem; the optimal combination can be inferred by repeating the process of inferring which one of multiple combination targets should be selected next.

[0075] The encoder part of the inference model 311, which is a transformer model, may be configured as shown in Fig. 7. Fig. 7 is a diagram showing an example of the configuration of the encoder included in the inference model 311. The encoder shown in the figure is F A and F B It consists of two parts:

[0076] Even if the route connects the same points, the transit cost may be different if the departure point and destination are reversed. For example, even if the same route is taken, the required travel time may be longer when heading to the center of a city than when heading to the suburbs. Details will be explained below, but F A and F B By using these two encoders together, it becomes possible to perform optimization taking into account such asymmetry of the route.

[0077] F A and F B Both of them take q (query), k (key), and v (value) as input. A and F B It consists of L layers, each consisting of two sublayers: a multi-head attention layer and a feedforward network layer (each with a residual connection and normalization layer). By using a multi-layer encoder, it is possible to guide learning so that local features are extracted in the lower layers and global features are extracted in the higher layers.

[0078] Although not shown, the decoder of the inference model 311 is also composed of L layers. Each layer in the decoder includes two sub-layers of the encoder, as well as a multi-head attention layer (and a residual connection / normalization layer) that receives the output from the first sub-layer. The decoder of the inference model 311 can be configured similarly to the decoder in a Transformer model that processes natural language.

[0079] When the learning unit 306 performs reinforcement learning of the inference model 311, it is sufficient to provide information (for example, coordinates) indicating each point to be delivered and each route connecting these points as input data. This input data can be said to be data with a graph structure in which each of the above points is a node and each of the above routes is an edge. Therefore, hereinafter, the input data used for learning will be referred to as graph data. For example, points a1 to a M From any of the M points up to point b1 to b N The route to any of the N points can be represented by graph data consisting of nodes indicating each of these points and edges indicating the route between each of the points.

[0080] When the graph data is input to the encoder of the inference model 311, the data acquisition unit 301 acquires it and passes it to the inference unit 302. Then, the inference unit 302 calculates the distance between points a1 and a M , b1~b N The embedding vectors of each node corresponding to the input vectors are calculated and input to the multi-head attention layer of the encoder shown in Figure 7.

[0081] At this time, the inference unit 302 adds position information indicating the positional relationship (or order) of the input nodes to each embedding vector. This represents the edges between the nodes. Note that the inference unit 302 may calculate an embedding vector corresponding to each edge and input it to the multi-head attention layer.

[0082] The inference unit 302 also inputs information D(M×N) indicating the cost incurred when passing through the route, such as the travel distance, as information about the route between each point, i.e., the edge in the graph data, to the multi-head attention layer. Here, in the inference model 311 shown in FIG. 7, F A D(M×N) is input to F B is the transpose of D, T (N×M) is entered. This allows F A At points a1 to a M From any of the points b1 to b NThis makes it possible to extract features that take into account the cost of moving to one of the B At points b1 to b N From any of the points a1 to a M In this way, by using two encoders in combination, it becomes possible to perform optimization taking into account the asymmetry of the route. During learning, the learning unit 306 only needs to learn, for each route connecting each target point, the cost of traveling along the route in a first direction and the cost of traveling along the route in a second direction opposite to the first direction.

[0083] In addition to costs such as travel distance, area names and time information can also be incorporated into the attention mechanism as edge information. This makes it possible to incorporate realistic structures into the encoder, rather than simply using simplified mathematical expressions.

[0084] Here, an attention mechanism is provided in the multi-head attention layer of the encoder of the inference model 311. Therefore, the encoder of the inference model 311 can extract features that take into account the structure of the entire graph, not just the information of adjacent nodes. Specifically, the attention mechanism calculates the attention of each node, and updates the feature of the node based on the feature of other nodes with which it is highly related. This allows the feature of the entire graph to be reflected in the feature of each node.

[0085] On the other hand, the decoder of the inference model 311 infers the next node to be selected, in other words, the next destination, based on the features extracted by the encoder. The decoder is trained (each parameter included in the decoder is updated) by reinforcement learning so that it can select the node that maximizes or minimizes the objective function.

[0086] In decoder training, it is preferable to use masking to exclude nodes that do not satisfy the conditions to be satisfied in optimization so that such nodes are not selected. This reduces the possibility of inferring a solution that does not satisfy the conditions, which can occur in probabilistic models, and enables efficient training of a high-performance model. The above conditions may be, for example, optimization constraints (e.g., not passing through the same point, collecting at a fixed collection time, not collecting more than the truck's capacity limit, etc.), or may be other conditions.

[0087] As described above, when the inference model 311, which is a Transformer model, is applied, the attention mechanism reflects the features of the entire graph data in the features of each node, making it possible to infer an appropriate solution by taking into account the structure of the entire graph data.

[0088] Furthermore, as described above, when the inference model 311 is a Transformer model, the Transformer model may be a model trained by inputting data indicating each of a plurality of points as an embedding vector into an attention layer, and inputting data indicating the cost of passing through each route connecting the plurality of points into the attention layer. This provides the effect of being able to infer an appropriate solution by appropriately considering the cost of passing through a plurality of points and each route connecting them, in addition to the effect provided by the information processing device 1 of exemplary embodiment 1.

[0089] The encoder of the inference model 311 may include a structure such as a recurrent neural network (RNN) or a long short-term memory (LSTM). This is true whether the inference model 311 is a transformer model or another model.

[0090] Furthermore, as described above, the inference model 311 may be a model that has learned, for each route, the cost of traveling along the route in a first direction and the cost of traveling along the route in a second direction opposite to the first direction. This provides, in addition to the effects of the information processing device 1 of exemplary embodiment 1, an effect that, when the cost differs depending on the direction along the route, it is possible to infer an appropriate solution that takes this difference into account.

[0091] (Example of inference) FIG. 8 is a diagram showing an example of inferring an optimal delivery route using an inference model including the encoder shown in FIG. 7. Specifically, in the example of FIG. 8, input data including edge information D(N×N) is first input to the encoder, and emb a1 ~emb aN and emb b1 ~emb bN The following feature values ​​are extracted. Note that N is the number of points to be targeted for delivery route generation.

[0092] Next, the emb extracted by the encoder a1 ~emb aN Among the features of the starting point s(1) of the delivery route, e(1) is input to the decoder as q (query). a_s(i) In addition, the decoder has emb, which is the feature of each point that can be reached after the starting point s(1). b1 ~emb bN is input as k (key) and v (value).

[0093] In addition, these feature quantities input to the decoder may be subjected to masking. For example, N If there is a point in the list that does not satisfy the specified constraints, the feature value corresponding to that point can be masked to prevent that point from being selected as the next destination. If there is no such point, the mask values ​​can be set to all zero as shown in the figure.

[0094] By inputting the above data into the decoder, the decoder outputs the probability that each point that can be reached after the starting point s(1) is the next point to be reached. In Figure 2, this probability is p i (i=1 to N). The decoder may output, for example, the position where this probability is maximum as the point s(2) to go to next after the starting point s(1).

[0095] Next, the decoder receives the feature e(1) of the starting point s(1) and the feature e(2) of the next destination point s(2) as inputs as q, and also receives the feature emb of each possible destination point after the starting point s(1). b1 ~emb bN are input to the decoder as k and v. Here, since point s(2) is already selected, masking is performed on the feature corresponding to point s(2).

[0096] As a result, the decoder outputs the probability that, for each possible next point after proceeding from the starting point s(1) to point s(2), that point is the point to proceed to.The decoder may then output, for example, the position with the highest probability as point s(3) to proceed to after point s(2).By repeating the same process thereafter, a delivery route from the starting point s(1) to the arrival point (N-1) is generated as shown in the figure.

[0097] The presentation unit 303 may present the inference result of the inference unit 302 by presenting a completed delivery route such as ImgN shown in FIG. 8. The presentation unit 303 may also present to the user the progress of the delivery route until it is completed. For example, the presentation unit 303 may display Img2 shown in FIG. 8 when point s(2), the next destination point after point s(1), is estimated. The presentation unit 303 may then update Img2 to Img3 shown in FIG. 8 when point s(3), the next destination point after point s(2), is estimated. This makes it possible to check the generated portion of the delivery route and start delivery along the generated portion during the period until the delivery route to the arrival point (N-1) is completed.

[0098] Furthermore, instead of outputting the point corresponding to the highest probability, the decoder may output each point corresponding to a predetermined number of the highest probabilities, or each point whose probability is equal to or exceeds a predetermined upper limit. In this case, if the decoder outputs multiple points, the user can select which of these points to use as the next destination. This allows the generation of an optimal delivery route while taking into account the user's wishes.

[0099] (Additional learning of inference model 311) By performing additional learning on the trained inference model 311, it becomes possible to generate a reasonable delivery route that is suited to the area that is the target of the delivery plan. This will be explained based on FIG. 9. FIG. 9 is a diagram showing an example of additional learning of the inference model 311. FIG. 9 also shows the encoder and decoder included in the inference model 311. The encoder extracts the feature amount of the input data x, and the decoder generates the inference result a of the solution to the delivery plan based on the extracted feature amount. 1:N Output.

[0100] In the additional learning, for example, among the features output by the encoder, a general feature h that is commonly applied to various delivery plans is compared with a feature h that is applied to a specific region. AThis makes the inference model 311 more suitable for generating a delivery plan for the specific area.

[0101] In this case, the learning unit 306 changes the part of the output layer of the encoder that outputs the feature value h to the feature value h A Then, as shown in the figure, additional learning can be performed using data N containing the features of the specific region. This allows the feature h output by the encoder to be converted to the feature h A can be changed to.

[0102] Similarly, the learning unit 306 can also add a feature extracted by the encoder. In this case, the learning unit 306 adds a new feature h A After modifying the output layer of the encoder to output A Additional learning can be performed using data related to

[0103] For example, consider the case where an inference model 311 generated for a certain region is adapted to a region with a different city size. To give a specific example, an inference model 311 trained with high accuracy for approximately 100 locations can be adapted to generate delivery plans for a larger city with approximately 1,000 locations. In this case, additional learning can be performed by adding features conditioned on city size. The simplest approach is to add features for changing the city. Alternatively, new features can be additionally learned using the city size and previously trained features as inputs. Similarly, the number of trucks available and user usage status (e.g., redelivery and reservation information) vary from region to region, and such information can be added as features for additional learning.

[0104] In this way, the learning unit 306 can change the internal representation to suit the user's needs by adding new features to the features output by the encoder or by performing adjustment processing to change the features. The internal representation output by the encoder represents the inherent features of the input data, but adding complementary features depending on the purpose of the problem can sometimes result in a more appropriate internal representation.

[0105] As described above, when adding new features to the internal representation output by the encoder, the learning unit 306 simply adds a new feature generation layer to the encoder and then performs additional learning (additional learning process). The new feature generation layer generates new features that take into account the characteristics of the problem, such as the objective function and constraints that reflect the user's needs.

[0106] For example, in a delivery planning problem, features corresponding to the distance between each point, features corresponding to the vehicle's load capacity limit, features such as text explaining the user's individual circumstances, etc. may be added to the features output by the encoder. By additionally training the encoder to output features generated from region-specific data tailored to the user's needs, it becomes possible to generate delivery plans that suit the region and the user's needs.

[0107] Furthermore, in additional learning or re-learning, it is preferable to change the masking process of the decoder to match the conditions that must be met. This allows for efficient learning of the conditions that must be met. For example, having the delivery pick-up time be in the evening instead of the morning should be incorporated into the model as a hard constraint. Also, additional constraints that must be observed, such as the number of available trucks or the working hours of drivers, may be added. In such cases, by learning while appropriately changing the target of the masking process in the decoder, it becomes possible to quickly reflect the constraints that must be observed in the model.

[0108] (Processing flow) The flow of processing executed by the information processing device 3 will be described with reference to Fig. 10. Fig. 10 is a flow diagram showing the flow of processing executed by the information processing device 3. The flow in Fig. 10 includes each step of the inference method and the inference model generation method according to this exemplary embodiment. Note that, below, an example will be described in which the information processing device 3 solves an optimization problem of inferring an optimal route that passes through each of a plurality of target points while satisfying predetermined conditions.

[0109] In S31 (data acquisition process), the data acquisition unit 301 acquires input data related to a combinatorial optimization problem for which a solution is to be obtained. The method for acquiring the input data is not particularly limited. For example, the data acquisition unit 301 may acquire the input data via the communication unit 32 or the input unit 33.

[0110] In S32 (inference processing), the inference unit 302 infers a solution corresponding to the input data acquired in S31, specifically an optimal route, using the inference model 311. As described above, the inference model 311 is an inference model generated by reinforcement learning. The inference model 311 is an inference model for inferring a solution to an optimization problem, and includes an encoder that extracts features of the input data and a decoder that generates information indicating the solution to the optimization problem using the features extracted by the encoder.

[0111] In S33, the presentation unit 303 presents to the user the inference result of S32, i.e., the optimal route inferred by the inference unit 302. For example, the presentation unit 303 may present the inferred optimal route to the user by causing the terminal device 4 to display an image showing the inferred optimal route on a map, as in the example of FIG.

[0112] In S34, the correction unit 304 determines whether or not the route presented in S33 is to be the confirmed route. If it is determined that the route is to be the confirmed route (YES in S34), the process proceeds to S36. On the other hand, if it is determined that the route is not to be the confirmed route (NO in S34), the process proceeds to S35. For example, the correction unit 304 may determine NO in S34 if a predetermined correction operation is detected on the route presented in S33.

[0113] In S35, the correction unit 304 corrects the route presented in S33. After the route has been corrected, the process returns to S33, and the presentation unit 303 presents the corrected route as the optimal route. The route may be corrected according to a user instruction, or may be automatically corrected by the correction unit 304. In the latter case, the correction unit 304 may determine the content of the correction based on, for example, a rule base.

[0114] In S36, the guiding unit 305 executes route guidance along the route determined in S34. After the execution of route guidance is completed, that is, after the user has finished moving along the determined route, the process proceeds to S37.

[0115] In S37, the learning unit 306 evaluates the route determined in S34. In evaluating the route, the learning unit 306 calculates a reward using a predetermined reward function based on the time required between each point in the user's movement along the route, and calculates an evaluation value for the entire route based on the calculated reward.

[0116] In S38 (update processing), the learning unit 306 updates the inference model 311 by reinforcement learning based on the evaluation result of S37. In this way, the learning unit 306 may perform reinforcement learning after completing movement along the inferred optimal route. This type of reinforcement learning is called offline reinforcement learning. This completes the processing in FIG. 10. Note that when offline reinforcement learning is applied, the processing in S37 and S38 may be performed on multiple optimal routes obtained by inference after multiple inferences of optimal routes have been performed.

[0117] The learning unit 306 may also perform online reinforcement learning. In this case, in S32, the inference unit 302 infers the next point to go to instead of the entire route, and the learning unit 306 may calculate a reward for the route up to that point when the point is reached. In this case, such processing is repeated until the final point is reached, and the learning unit 306 evaluates the entire route when the final point is reached and updates the inference model 311.

[0118] As described above, when the quality of the inferred solution is evaluated, a reward is calculated based on the result, and the calculated reward is used to update the inference model 311, the inference accuracy of the inference model 311 can be continuously improved by continuously using the information processing device 3. This makes it possible to obtain higher quality inference results each time the information processing device 3 is used.

[0119] [Software implementation example] Some or all of the functions of the information processing devices 1 and 3 may be realized by hardware such as an integrated circuit (IC chip), or by software.

[0120] In the latter case, the information processing devices 1 and 3 are realized by, for example, a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in Fig. 11. Fig. 11 is a block diagram showing the hardware configuration of computer C that functions as information processing device 1 or 3.

[0121] The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P (inference program / learning program) for operating the computer C as the information processing device 1 or 3. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing each function of the information processing device 1 or 3.

[0122] The processor C1 may be, for example, a central processing unit (CPU), a graphic processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a tensor processing unit (TPU), a quantum processor, a microcontroller, or a combination thereof. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0123] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may also include a communication interface for transmitting and receiving data to and from other devices. The computer C may also include an input / output interface for connecting input / output devices such as a keyboard, mouse, display, and printer.

[0124] Furthermore, the program P can be recorded on a non-transitory tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can acquire the program P via such a recording medium M. The program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also acquire the program P via such a transmission medium.

[0125] Furthermore, each of the above functions of information processing device 1 or 3 may be realized by a single processor provided in a single computer, by multiple processors provided in a single computer working in cooperation, or by multiple processors provided in each of multiple computers working in cooperation. Furthermore, the program for causing information processing device 1 or 3 to realize each of the above functions may be stored in a single memory provided in a single computer, or may be distributed and stored in multiple memories provided in a single computer, or may be distributed and stored in multiple memories provided in each of multiple computers.

[0126] [Appendix A] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0127] (Appendix A1) an information processing device comprising: a data acquisition means for acquiring input data related to a combinatorial optimization problem for which a solution is sought; and an inference means for inferring a solution corresponding to the input data using an inference model for inferring a solution to the optimization problem, the inference model being generated by reinforcement learning, the inference model including an encoder for extracting features of the input data and a decoder for generating information indicating the solution to the optimization problem using the features extracted by the encoder.

[0128] (Appendix A2) The information processing device according to claim A1, wherein the data acquiring means acquires the input data including at least one of text data describing the optimization problem in a natural language and image data related to the optimization problem.

[0129] (Appendix A3) The information processing device according to claim A1 or A2, wherein the decoder is trained to generate at least one of image data indicating a solution to the optimization problem and text data indicating a solution to the optimization problem.

[0130] (Appendix A4) An information processing device described in any of Appendices A1 to A3, wherein the inference model is a model generated by learning about multiple target locations and each route between the locations, and infers an optimal route that passes through each of the locations while satisfying specified conditions.

[0131] (Appendix A5) The information processing device described in Appendix A4, wherein the inference model is a Transformer model, and the Transformer model is a model trained by inputting data indicating each of the multiple locations as an embedding vector into an attention layer, and inputting data indicating the cost of traveling each route connecting the multiple locations into the attention layer.

[0132] (Appendix A6) The information processing device described in Appendix A4 or A5, wherein the inference model is a model that has learned, for each route, the cost of traveling along the route in a first direction and the cost of traveling along the route in a second direction opposite to the first direction.

[0133] (Appendix A7) An information processing device comprising: an inference means for inferring a solution according to input data using an inference model that infers a solution to a combinatorial optimization problem, the inference model including an encoder that extracts features of input data input to the inference model and a decoder that generates information indicating a solution to the optimization problem using the features extracted by the encoder; and a learning means for updating the inference model by reinforcement learning based on evaluation results of the solution obtained by the inference means.

[0134] (Appendix A8) The information processing device described in Appendix A7, wherein the learning means performs an adjustment process to change some of the features to be extracted by the encoder or to add features to be extracted by the encoder, and additionally learns the inference model after the adjustment process.

[0135] [Appendix B] (Appendix B1) An inference method comprising: a data acquisition process in which at least one processor acquires input data related to a combinatorial optimization problem for which a solution is to be sought; and an inference process in which an inference model for inferring a solution to the optimization problem, the inference model being generated by reinforcement learning, the inference model including an encoder that extracts features of the input data and a decoder that generates information indicating the solution to the optimization problem using the features extracted by the encoder, to infer a solution corresponding to the input data.

[0136] (Appendix B2) The inference method described in Appendix B1, wherein in the data acquisition process, the at least one processor acquires the input data including at least one of text data describing the optimization problem in a natural language and image data related to the optimization problem.

[0137] (Appendix B3) The inference method of appendix B1 or B2, wherein the decoder is trained to generate at least one of image data representing a solution to the optimization problem and text data representing a solution to the optimization problem.

[0138] (Appendix B4) An inference method described in any of Appendices B1 to B3, wherein the inference model is a model generated by learning about multiple target locations and each route between the locations, and infers the optimal route that passes through each of the locations while satisfying specified conditions.

[0139] (Appendix B5) An inference method described in Appendix B4, wherein the inference model is a Transformer model, and the Transformer model is a model trained by inputting data indicating each of the multiple locations as an embedding vector into an attention layer, and inputting data indicating the cost of traveling each route connecting the multiple locations into the attention layer.

[0140] (Appendix B6) An inference method described in Appendix B4 or B5, wherein the inference model is a model that has learned, for each route, the cost of traveling along the route in a first direction and the cost of traveling along the route in a second direction opposite to the first direction.

[0141] (Appendix B7) A method for generating an inference model, comprising: an inference process in which at least one processor infers a solution to a combinatorial optimization problem using an inference model that includes an encoder that extracts features of input data input to the inference model and a decoder that generates information indicating a solution to the optimization problem from the features extracted by the encoder; and an update process in which the inference model is updated by reinforcement learning based on evaluation results of the solution obtained in the inference process.

[0142] (Appendix B8) A method for generating an inference model described in Appendix B7, including an adjustment process that changes some of the features to be extracted by the encoder or adds features to be extracted by the encoder, and an additional learning process that additionally learns the inference model after the adjustment process.

[0143] [Appendix C] (Appendix C1) An inference program that causes a computer to function as inference means that infers a solution based on input data using an inference model that includes: data acquisition means that acquires input data related to a combinatorial optimization problem for which a solution is to be sought; and an inference model for inferring a solution to the optimization problem, the inference model being generated by reinforcement learning, the inference model including an encoder that extracts features of the input data and a decoder that generates information indicating the solution to the optimization problem using the features extracted by the encoder.

[0144] (Appendix C2) The inference program according to Appendix C1, wherein the data acquisition means acquires the input data including at least one of text data describing the optimization problem in a natural language and image data related to the optimization problem.

[0145] (Appendix C3) An inference program as described in Appendix C1 or C2, wherein the decoder is trained to generate at least one of image data indicating a solution to the optimization problem and text data indicating a solution to the optimization problem.

[0146] (Appendix C4) An inference program described in any of Appendices C1 to C3, wherein the inference model is a model generated by learning about multiple target locations and each route between the locations, and infers an optimal route that passes through each of the locations while satisfying specified conditions.

[0147] (Appendix C5) An inference program described in Appendix C4, wherein the inference model is a Transformer model, and the Transformer model is a model trained by inputting data indicating each of the multiple locations as an embedding vector into an attention layer, and inputting data indicating the cost of traveling each route connecting the multiple locations into the attention layer.

[0148] (Appendix C6) An inference program described in Appendix C4 or C5, wherein the inference model is a model that has learned, for each route, the cost of traveling along the route in a first direction and the cost of traveling along the route in a second direction opposite to the first direction.

[0149] (Appendix C7) A learning program that causes a computer to function as an inference means that infers a solution based on input data using an inference model that infers a solution to a combinatorial optimization problem, the inference model including an encoder that extracts features of input data input to the inference model and a decoder that generates information indicating a solution to the optimization problem using the features extracted by the encoder, and an update means that updates the inference model by reinforcement learning based on the evaluation results of the solution obtained by the inference means.

[0150] (Appendix C8) A learning program described in Appendix C7, which causes the computer to perform an adjustment process to change some of the features to be extracted by the encoder or to add features to be extracted by the encoder, and also performs an additional learning process to additionally learn the inference model after the adjustment process.

[0151] [Appendix D] (Appendix D1) An information processing device comprising at least one processor that executes: a data acquisition process that acquires input data related to a combinatorial optimization problem for which a solution is to be found; and an inference process that infers a solution corresponding to the input data using an inference model for inferring a solution to the optimization problem, the inference model being generated by reinforcement learning, the inference model including an encoder that extracts features of the input data and a decoder that generates information indicating a solution to the optimization problem using the features extracted by the encoder.

[0152] The information processing device may further include a memory, and the memory may store an inference program for causing the at least one processor to execute each of the processes.

[0153] (Appendix D2) The information processing device described in Appendix D1, wherein in the data acquisition process, the at least one processor acquires the input data including at least one of text data describing the optimization problem in a natural language and image data related to the optimization problem.

[0154] (Appendix D3) The information processing device according to claim D1 or D2, wherein the decoder is trained to generate at least one of image data indicating a solution to the optimization problem and text data indicating a solution to the optimization problem.

[0155] (Appendix D4) An information processing device described in any of Appendices D1 to D3, wherein the inference model is a model generated by learning about multiple target locations and each route between the locations, and infers the optimal route that passes through each of the locations while satisfying specified conditions.

[0156] (Appendix D5) The information processing device described in Appendix D4, wherein the inference model is a Transformer model, and the Transformer model is a model trained by inputting data indicating each of the multiple locations as an embedding vector into an attention layer, and inputting data indicating the cost of traveling each route connecting the multiple locations into the attention layer.

[0157] (Appendix D6) An information processing device described in Appendix D4 or D5, wherein the inference model is a model that has learned, for each route, the cost of traveling along the route in a first direction and the cost of traveling along the route in a second direction opposite to the first direction.

[0158] (Appendix D7) An information processing device comprising at least one processor, the at least one processor executing an inference process to infer a solution according to input data using an inference model that infers a solution to a combinatorial optimization problem, the inference model including an encoder that extracts features of input data input to the inference model and a decoder that generates information indicating a solution to the optimization problem from the features extracted by the encoder, and an update process to update the inference model by reinforcement learning based on evaluation results of the solution obtained by the inference process.

[0159] The information processing device may further include a memory, and the memory may store a learning program for causing the at least one processor to execute each of the processes.

[0160] (Appendix D8) The information processing device described in Appendix D7, wherein the at least one processor performs an adjustment process that changes some of the features to be extracted by the encoder or adds features to be extracted by the encoder, and an additional learning process that additionally learns the inference model after the adjustment process.

[0161] [Appendix E] (Appendix E1) A non-transitory recording medium having recorded thereon an inference program that causes a computer to execute: a data acquisition process that acquires input data related to a combinatorial optimization problem for which a solution is to be sought; and an inference process that infers a solution based on the input data using an inference model for inferring a solution to the optimization problem, the inference model being generated by reinforcement learning, the inference model including an encoder that extracts features of the input data and a decoder that generates information indicating the solution to the optimization problem using the features extracted by the encoder.

[0162] (Appendix E2) A non-transient recording medium having recorded thereon a learning program that causes a computer to execute an inference process for inferring a solution based on input data using an inference model that infers a solution to a combinatorial optimization problem, the inference model including an encoder that extracts features of input data input to the inference model and a decoder that generates information indicating a solution to the optimization problem from the features extracted by the encoder, and an update process for updating the inference model through reinforcement learning based on the evaluation results of the solution obtained in the inference process. [Explanation of symbols]

[0163] 1. Information processing equipment 11 Data acquisition unit (data acquisition means) 12 Reasoning part (reasoning means) 3. Information processing equipment 301 Data acquisition unit (data acquisition means) 302 Reasoning part (reasoning means) 311 Inference Model

Claims

1. a data acquisition means for acquiring input data related to a combinatorial optimization problem for which a solution is to be sought; an inference means for inferring a solution according to the input data using an inference model for inferring a solution to the optimization problem, the inference model being generated by reinforcement learning, the inference model including an encoder that extracts features of the input data and a decoder that generates information indicating a solution to the optimization problem using the features extracted by the encoder.

2. The information processing apparatus according to claim 1 , wherein the data acquiring means acquires the input data including at least one of text data describing the optimization problem in a natural language and image data relating to the optimization problem.

3. The information processing device according to claim 1 , wherein the decoder is trained to generate at least one of image data representing a solution to the optimization problem and text data representing a solution to the optimization problem.

4. The information processing device described in claim 1 or 2, wherein the inference model is a model generated by learning about multiple target locations and each route between the locations, and infers an optimal route that passes through each of the locations while satisfying specified conditions.

5. the inference model is a Transformer model; 5. The information processing device according to claim 4, wherein the Transformer model is a model trained by inputting data indicating each of the plurality of points as an embedding vector into an attention layer, and inputting data indicating the cost of traveling each route connecting the plurality of points into the attention layer.

6. The information processing device described in claim 4, wherein the inference model is a model that has learned, for each route, the cost of traveling along the route in a first direction and the cost of traveling along the route in a second direction opposite to the first direction.

7. At least one processor an inference process for inferring a solution according to input data using an inference model for inferring a solution to a combinatorial optimization problem, the inference model including an encoder for extracting features of input data input to the inference model, and a decoder for generating information indicating a solution to the optimization problem using the features extracted by the encoder; A method for generating an inference model, comprising: an update process for updating the inference model through reinforcement learning based on the evaluation results of the solution obtained in the inference process.

8. an adjustment process of changing some of the features to be extracted by the encoder or adding features to be extracted by the encoder; A method for generating an inference model as described in claim 7, including an additional learning process for additionally learning the inference model after the adjustment process.

9. At least one processor a data acquisition process for acquiring input data related to a combinatorial optimization problem for which a solution is sought; an inference process for inferring a solution according to the input data using an inference model for inferring a solution to the optimization problem, the inference model being generated by reinforcement learning, the inference model including an encoder that extracts features of the input data and a decoder that generates information indicating a solution to the optimization problem using the features extracted by the encoder.

10. Computer, a data acquisition means for acquiring input data related to a combinatorial optimization problem for which a solution is sought; and An inference program that functions as an inference means for inferring a solution according to input data using an inference model for inferring a solution to the optimization problem, the inference model being generated by reinforcement learning and including an encoder that extracts features of the input data and a decoder that generates information indicating a solution to the optimization problem using the features extracted by the encoder.

Images