Information processing device and information processing method

Abstract

This improves the accuracy and speed of solving large-scale combinatorial optimization problems by dividing them into subproblems. [Solution] The information processing device 100 for processing combinatorial optimization problems includes: a graph creation unit 112 that creates one or more subgraphs from a main graph; a mathematical optimization unit 115 that solves the combinatorial optimization problem for each subgraph using a mathematical optimization solver; a machine learning unit 117 that trains each subGNN so that the output of the subGNN corresponding to each subgraph is close to the solution of the mathematical optimization solver; a feature vector assignment unit 118 that assigns the feature vectors at each vertex of the subGNN obtained as a result of training to the corresponding vertices of the main GNN as input to the feature vectors of the main GNN corresponding to the graph data of the main graph; and a solution output unit 119 that outputs the solution obtained as a result of training the main GNN by setting a loss function so that the machine learning unit 117 solves the combinatorial optimization problem for the main graph.

Description

[Technical Field]

[0001] This invention relates to information processing technology for solving optimization problems. [Background technology]

[0002] Combinatorial optimization problems exist in various fields of real-world applications, such as shift scheduling optimization and delivery planning optimization. While there are numerous methods for solving combinatorial optimization problems, including linear programming, constraint programming, simulated annealing, genetic algorithms, and greedy algorithms, mathematical optimization solvers implementing these algorithms have limitations on the problem size they can solve within realistic computation timeframes. Even Ising machines, hardware specifically designed for solving combinatorial optimization problems, have limitations on the number of variables they can implement, and thus on the problem size they can handle. This also applies to quantum computers, which are hoped to become a reality in the future; given the limited number of qubits, the size of combinatorial optimization problems they can handle is also limited.

[0003] However, real-world combinatorial optimization problems are often extremely large in scale, making it difficult to directly address them using the aforementioned solving methods and hardware. Therefore, when dealing with large-scale combinatorial optimization problems, it is sometimes necessary to reduce the problem size before solving it, and in doing so, it is desirable to avoid as much as possible a decrease in solution accuracy and an increase in the solving time.

[0004] Regarding the processing of the combinatorial optimization problem described above, Patent Document 1 states: "In one aspect, the present invention aims to provide an information processing system, an information processing method, and a program that improves the ability to solve problems." "In one aspect, an information processing system is provided that searches for a solution to a problem represented by an energy function including a plurality of state variables. The information processing system has a plurality of nodes to which each of a plurality of subproblems generated by dividing the problem is assigned. Each of the plurality of nodes searches for a partial decomposition represented by a group of state variables from among the plurality of state variables that corresponds to the subproblem assigned to the node, and holds a plurality of solutions, including a first solution to the problem, that reflect the partial decomposition. The first node of the plurality of nodes transmits at least one of the plurality of solutions held by the first node to the second node of the plurality of nodes. The second node updates at least a portion of the plurality of solutions held by the second node based on the solution received from the first node." [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2022-6994 [Overview of the project] [Problems that the invention aims to solve]

[0006] Patent Document 1 proposes a method in which a large-scale combinatorial optimization problem is divided, solutions are searched for in each subproblem, and the overall solution is constructed by combining the partial decompositions, repeatedly sending and receiving partial decompositions between the subproblems until the termination condition is met, and the solution is repeatedly sought to minimize the objective function value in the original problem. However, this method has the problem that the number of iterations required to obtain a good overall solution becomes large and the computation time becomes long. Also, a simple combination of partial decompositions does not necessarily give a good overall solution.

[0007] The present invention has been made in view of such a background, and aims to improve the solution accuracy and the solving speed in combinatorial optimization problems, particularly in large-scale combinatorial optimization problems.

Means for Solving the Problem

[0008] To solve the above problems, in the present invention, one or more subgraphs are created from the main graph indicating the combinatorial optimization problem, and the solution is realized using the result of learning so that the solution of the mathematical optimization solver becomes close, taking advantage of the high accuracy of the mathematical optimization solver. In creating this subgraph, it is desirable to reduce (downsize) the main graph.

[0009] More specifically, an information processing apparatus for processing a combinatorial optimization problem that can be defined on a graph, comprising: a problem data acquisition unit to which problem data is input; a graph creation unit that creates one or more subgraphs obtained by reducing the main graph, which is the graph to be solved, by referring to the problem data; a mathematical optimization unit that solves the combinatorial optimization problem for each of the subgraphs using a mathematical optimization solver; a machine learning unit that learns each of the sub GNNs using the solution obtained by the mathematical optimization solver as teacher data so that the output of each sub GNN corresponding to each subgraph becomes close to the solution of the mathematical optimization solver; a feature vector assignment unit that assigns the feature vector at each vertex of the sub GNN obtained as a result of learning to each vertex of the corresponding main GNN as an input of the feature vector of the main GNN corresponding to the graph data of the main graph; and a solution output unit that outputs the solution obtained as a result of setting a loss function and learning the main GNN so that the machine learning unit solves the combinatorial optimization problem for the main graph.

[0010] The present invention also includes an information processing method executed by the above information processing apparatus, a program for causing the information processing apparatus to function, and a storage medium storing the program.

Effects of the Invention

[0011] According to the present invention, for combinatorial optimization problems, high-precision and high-speed solution can be achieved. Problems, configurations, and effects other than those described above will be clarified by the description of the embodiments for implementing the following invention.

Brief Description of the Drawings

[0012] [Figure 1] A block diagram showing an example of the schematic hardware configuration of the information processing apparatus 100 in an embodiment of the present invention. [Figure 2] A functional block diagram showing an example of the main functional configuration of the information processing apparatus 100 in an embodiment of the present invention. [Figure 3] A flowchart showing an example of the procedure of the optimization process in an embodiment of the present invention. [Figure 4] A diagram showing an example of a solution to the maximum independent set problem in an embodiment of the present invention. [Figure 5] A diagram showing an example of the process of creating a subgraph by graph compression in an embodiment of the present invention. [Figure 6] A diagram showing an example of the process of creating a subgraph by graph partitioning in an embodiment of the present invention. [Figure 7] A flowchart showing an example of the procedure of solving the combinatorial optimization problem on a subgraph using a mathematical optimization solver in an embodiment of the present invention. [Figure 8] A flowchart showing an example of the procedure of selecting the mathematical optimization solver to be used (details of processing step S33) in an embodiment of the present invention. [Figure 9] A diagram showing an example of a graph neural network (GNN) in an embodiment of the present invention. [Figure 10] A flowchart showing an example of the learning procedure of the sub-GNN (details of processing step S204) in an embodiment of the present invention. [Figure 11] A flowchart showing an example of the procedure of creating the loss function of the sub-GNN (details of processing steps S42 and S43) in an embodiment of the present invention. [Figure 12]A figure showing comparative examples of the learning processes of sub-GNNs with different loss functions in one embodiment of the present invention. [Figure 13] A diagram illustrating an example of the procedure for assigning feature vectors from a subgraph to the main graph in one embodiment of the present invention. [Figure 14] This figure shows an example of a case where the loss function of the main GNN in one embodiment of the present invention is described entirely by quadratic equations. [Figure 15] This figure shows an example of how the loss function of the main GNN in one embodiment of the present invention can be described by separating it into a quadratic and a linear equation. [Figure 16] A figure showing a comparative example of the learning process of the main GNN with different loss functions in one embodiment of the present invention. [Figure 17] This figure shows an example of a procedure for rounding the output of continuous values ​​obtained as a result of learning the main graph in one embodiment of the present invention to binary values. [Figure 18] This figure shows an example of the impact on solution performance when a portion of the optimization process is omitted in one embodiment of the present invention. [Figure 19] This figure shows an example of the relationship between the number of subgraphs used for training and the solution accuracy of the main GNN in one embodiment of the present invention. [Modes for carrying out the invention]

[0013] Embodiments of the present invention will be described in detail below with reference to the drawings. In the following description, identical or similar components may be denoted by a common reference numeral, and redundant descriptions may be omitted. Also, if there are multiple elements having the same or similar function, the same reference numeral may be denoted by different subscripts. Furthermore, if it is not necessary to distinguish between multiple elements, subscripts may be omitted in the description.

[0014] The optimization problem solving process according to the present invention is applicable to many situations, but in this embodiment, we deal with the maximum independent set problem. The maximum independent set problem is the problem of finding the largest independent set in a given graph, and the accuracy of the solution can be evaluated by the size of the independent set. Here, an independent set in a given graph is a set of vertices in which no edges exist between any vertices within the set.

[0015] In this embodiment, the graph data in the combinatorial optimization problem is reduced to one or more subgraphs smaller in scale. A sub-GNN (Graph Neural Network) corresponding to each subgraph is trained so that the solution of the mathematical optimization solver is close to that of the sub-GNN. The main GNN is then trained based on the result of mapping the feature vectors at each node of the trained sub-GNN to the nodes of the main GNN corresponding to the graph data. For this purpose, the solution obtained by the mathematical optimization solver is used as training data. The details of this will be explained below. Note that "close to the solution of the mathematical optimization solver" means that the difference is within a predetermined condition. Here, the predetermined condition includes being within a predetermined rank and within a predetermined value. The difference includes simple differences, squared errors, etc.

[0016] Figure 1 is a block diagram showing an example of a schematic hardware configuration of the information processing device 100 in this embodiment. The information processing device 100 shown in Figure 1 performs a work plan optimization process as a combinatorial optimization problem. For this purpose, the information processing device 100 is configured to include a processor 101, main memory 102, auxiliary memory 103, input device 104, output device 105, communication device 106, one or more combinatorial optimization devices 107, one or more machine learning devices 108, and a system bus 109 that connects these devices in a communicative manner.

[0017] The information processing device 100 may be implemented using, for example, a virtual information processing resource such as a cloud server, which is provided in part or in whole by a cloud system. Alternatively, the information processing device 100 may be implemented by, for example, multiple information processing devices that operate in cooperation with each other and are connected in a communicative manner. In this case, the input device 104 and the output device 105 are implemented in a terminal device such as a PC connected to the information processing device 100. In this case, the information processing device 100 and the terminal device can be connected via a network such as the Internet. Furthermore, the information processing device 100 can be connected to multiple terminal devices. The various configurations of the information processing device 100 will be described below.

[0018] First, the processor 101 is composed of components such as a CPU (Central Processing Unit) and an MPU (Micro Processing Unit). Therefore, the processor 101 executes various processes according to the program described later. The main memory 102 is a device that temporarily stores programs and data for processing by the processor 101. Therefore, the main memory 102 can be implemented using, for example, ROM (Read Only Memory) (SRAM (Static Random Access Memory), NVRAM (Non-Volatile RAM), Mask ROM (Mask Read Only Memory), PROM (Programmable ROM), etc.), RAM (Random Access Memory) (DRAM (Dynamic Random Access Memory), etc.). Therefore, the program is stored in the aforementioned memory device or storage medium.

[0019] Furthermore, the auxiliary storage device 103 is a device for storing programs and data. For this reason, the auxiliary storage device 103 can be implemented as a hard disk drive, flash memory, SSD (solid state drive), optical storage device (CD (compact disc), DVD (digital versatile disc), etc.). The programs and data stored in the auxiliary storage device 103 are read into the main memory 102 as needed for processing by the processor 101.

[0020] Furthermore, the input device 104 is a user interface that accepts information input from the user. The input device 104 can be implemented as, for example, a keyboard, mouse, card reader, touch panel, etc. The output device 105 is a user interface that provides information to the user. The output device 105 can be implemented as, for example, a display device that visualizes various information (LCD (Liquid Crystal Display), graphics card, etc.), an audio output device (speaker), a printing device, etc. Note that the input device 104 and the output device 105 may be implemented in other terminal devices as described above. Furthermore, the input device 104 and the output device 105 may be configured as an integrated unit, such as a touch panel.

[0021] Furthermore, the communication device 106 is a communication interface that communicates with other devices such as terminal devices. For this reason, the communication device 106 can be implemented using, for example, a NIC (Network Interface Card), a wireless communication module, a USB (Universal Serial Interface) module, a serial communication module, etc.

[0022] Furthermore, the combinatorial optimization device 107 is a device that solves the input combinatorial optimization problem. The combinatorial optimization device 107 may be dedicated hardware specifically designed to execute metaheuristic algorithms, such as simulated annealing, or it may be dedicated hardware that performs solution search for an optimization problem expressed as an Ising model. However, the solution of the combinatorial optimization problem may be performed by having a general-purpose computing processor 101 function as the combinatorial optimization device 107, without providing a dedicated combinatorial optimization device 107.

[0023] Furthermore, the combinatorial optimization device 107 may be implemented in the form of an expansion card installed in the information processing device 100, such as a GPU (Graphics Processing Unit). In addition, the combinatorial optimization device 107 may be composed of hardware such as a CMOS (Complementary Metal Oxide Semiconductor) circuit, an FPGA (Field Programmable Gate Array), or an ASIC (Application Specific Integrated Circuit).

[0024] Furthermore, the machine learning device 108 is a device that performs machine learning calculations on the input neural network. Therefore, the machine learning device 108 may be responsible for at least one of the processes of training the neural network and the process of performing inference using the trained neural network and presenting the results.

[0025] Therefore, the machine learning device 108 may take the form of an expansion card installed in the information processing device 100, such as a GPU (Graphics Processing Unit). Alternatively, the machine learning device 108 can be implemented using a specially designed AI chip with hardware such as a CMOS (Complementary Metal Oxide Semiconductor) circuit, FPGA (Field Programmable Gate Array), or ASIC (Application Specific Integrated Circuit). However, machine learning may be performed by the processor 101 functioning as the machine learning device 108, without the need for a dedicated machine learning device 108.

[0026] The combinatorial optimization device 107 and the machine learning device 108 include a control device, a storage device, an interface for connecting to the system bus 109, and send and receive commands and information to and from the processor 101 via the system bus 109. The combinatorial optimization device 107 and the machine learning device 108 may be connected to other combinatorial optimization devices 107 via a communication line, for example, and may operate in cooperation with other combinatorial optimization devices 107.

[0027] Next, the functions of the information processing device 100 will be described. Figure 2 is a functional block diagram showing the main functional configuration (software configuration) of the information processing device 100 in this embodiment. The information processing device 100 shown in Figure 2 includes a problem data acquisition unit 111, a graph creation unit 112, a mathematical representation creation unit 113, an optimization solver selection unit 114, a mathematical optimization unit 115, a loss function creation unit 116, a machine learning unit 117, a feature vector assignment unit 118, a solution output unit 119, and a storage unit DB.

[0028] The functions of the problem data acquisition unit 111 to the feature vector assignment unit 118 are realized by the processor 101 reading and executing a program stored in the main memory 102, or by the combinatorial optimization device 107 or the machine learning device 108. The function of the solution output unit 119 can be realized by the output device 105 and / or the communication device 106. In addition to the above functions, the information processing device 100 may also include other functions such as an operating system, file system, device driver, and DBMS (Database Management System).

[0029] Furthermore, the memory unit DB comprises a problem data storage unit DB1, a graph data storage unit DB2, a mathematical representation data storage unit DB3, a partial decomposition data storage unit DB4, a loss function data storage unit DB5, a feature vector data storage unit DB6, and an arithmetic program storage unit DB7. Therefore, the memory unit DB can be implemented using either the main memory 102 or the auxiliary memory 103.

[0030] Furthermore, the problem data acquisition unit 111 acquires external information, which is information input from outside to set the problem to be solved and various conditions for solving. Preferably, the problem data acquisition unit 111 stores this information as problem data D1 in the problem data storage unit DB1 of the storage unit DB. The external information (problem data D1) includes, for example, the setting conditions for the combinatorial optimization problem to be solved, the setting of weights in the optimization index, the upper limit of the computation time, the target graph, the number of layers and the type of activation function in the graph neural network (GNN), the dimension of the feature vector assigned to each vertex, and the learning termination conditions. The external information is also received by the user via a user interface (input device, output device, communication device, etc.).

[0031] Furthermore, the graph creation unit 112 creates a graph from the problem data D1 stored in the problem data storage unit DB1. In addition to creating the original graph (main graph) that is actually to be solved from the problem data D1, the graph creation unit 112 also creates one or more subgraphs that are smaller in scale by downsizing the main graph. Note that downsizing includes compression and / or division of the main graph. The graph creation unit 112 then stores the created main graph and subgraphs as graph data D2 in the graph data storage unit DB2. Note that the graph can be stored in the form of an adjacency matrix, distance matrix, adjacency list, etc.

[0032] Furthermore, the mathematical representation creation unit 113 creates (formulates) mathematical representation data for the combinatorial optimization problem from the problem data D1 and graph data D2. It is desirable that the mathematical representation creation unit 113 then stores the created mathematical data as mathematical representation data D3 in the mathematical representation data storage unit DB3 of the storage unit DB. Here, if the objective function of the combinatorial optimization problem includes a quadratic expression, it is desirable that the mathematical representation creation unit 113 converts the quadratic terms, which are expressed by the square of the same binary variable, into linear terms while maintaining the coefficients of the quadratic terms, and then creates (converts) the mathematical representation of the objective function using a quadratic expression consisting only of intersection terms, which are products of different variables, and a linear expression including the converted linear terms. Details of this will be described later using mathematical formulas.

[0033] Furthermore, the optimization solver selection unit 114 selects an optimization solver to be used in solving the combinatorial optimization problem in the mathematical optimization unit 115 based on the problem data D1, the graph data storage unit DB2 containing graph data D2, and the mathematical representation data storage unit DB3 containing mathematical representation data D3.

[0034] Furthermore, the mathematical optimization unit 115 uses the combinatorial optimization solver selected by the optimization solver selection unit 114 to perform a solution on the mathematical representation data D3 of the combinatorial optimization problem. It is desirable that the mathematical optimization unit 115 then stores the obtained solution as partial data D4 in the partial data storage unit DB4.

[0035] Furthermore, the loss function creation unit 116 creates a loss function for machine learning from the problem data D1, graph data D2, mathematical representation data D3, and partial decomposition data D4. It is desirable that the loss function creation unit 116 then stores the created loss function as loss function data D5 in the loss function data storage unit DB5.

[0036] Furthermore, the machine learning unit 117 creates a GNN from the problem data D1 and graph data D2. Then, the machine learning unit 117 trains the GNN using the problem data D1 and loss function data D5. As a result, the machine learning unit 117 trains the sub-GNN using the partial decomposition data D4, which is the solution result from the mathematical optimization unit 115, as training data, so that the output of the sub-GNN corresponding to the subgraph of the graph data D2 is close to the solution of the mathematical optimization solver.

[0037] Furthermore, it is desirable for the machine learning unit 117 to store the GNN feature vectors obtained as a result of learning as feature vector data D6 in the feature vector data storage unit DB6. In addition, the machine learning unit 117 may output the variable values ​​obtained as a result of learning (solution to the combinatorial optimization problem) to the solution output unit 119.

[0038] Furthermore, the feature vector assignment unit 118 determines from the graph data D2 and feature vector data D6 which vertices of the main graph each vertex of the subgraph was created from. It is desirable that the feature vector assignment unit 118 then sets the initial input value of the feature vector for each vertex of the main GNN to a linear combination of the feature vectors obtained during the training of the sub-GNN for the group of vertices of the subgraph corresponding to each vertex of the main GNN.

[0039] Furthermore, the solution output unit 119 reads the solution obtained by the machine learning unit 117, calculates the objective function value and the number of constraint violations for the combinatorial optimization problem from the problem data D1, graph data D2, and mathematical representation data D3, and outputs them to the output device 105 and the communication device 106.

[0040] Furthermore, the calculation program storage unit DB7 stores the calculation program D7. Here, the calculation program D7 is a program that enables each part to function as follows. The graph creation unit 112 creates the main graph and subgraphs. • Mathematical representation creation unit 113 created the formalization. • The mathematical optimization unit 115 solves the combinatorial optimization problem. The loss function creation unit 116 creates the loss function. • Machine learning unit 117 performs machine learning using the given loss function. The solution output unit 119 calculates the objective function value and the number of constraint violations.

[0041] Next, the processing flow in this embodiment will be described. Figure 3 is a flowchart showing an example of the optimization process procedure in one embodiment of the present invention. The optimization process shown in Figure 3 is preferably executed by the functional units shown in Figure 2 using an information processing device 100 configured with the hardware shown in Figure 1. Hereinafter, the process executed by the information processing device 100 as shown in Figure 3 will be referred to as the optimization process S200. In the following, the letter "S" preceding the reference numerals means a processing step. The optimization process S200 is started, for example, by receiving instructions from the user via the input device 104.

[0042] The general flow of the optimization process S200 will be explained below with reference to Figure 3. First, in processing step S201, the problem data acquisition unit 111 accepts data input for a combinatorial optimization problem with a graph structure according to the user's operation. A combinatorial optimization problem with a graph structure is not limited to problems originally defined on a graph, such as the maximum independent set problem, the maximum cut problem, or the traveling salesman problem, but can be any problem in which the interaction between decision variables can be represented by a graph.

[0043] Furthermore, in processing step S201, the system may accept input from the user regarding the setting of weights in the optimization metrics, the upper limit of the computation time, the number of layers and type of activation function of the graph neural network (GNN) used in processing steps S204 and S205, the dimension of the feature vector assigned to each vertex, the learning termination conditions, and the method of assigning feature vectors in processing step S205. The problem data input by the user may be stored as problem data D1 in the problem data storage unit DB1 by the problem data acquisition unit 111.

[0044] Furthermore, in processing step S202, the graph creation unit 112 creates the graph to be created (main graph) by referring to the problem data D1 input in processing step S201. However, if the combinatorial optimization problem input in processing step S201 is a problem that is originally defined on a graph and the main graph is already input in the problem data D1, then it is not necessary to create the main graph. If the problem data D1 does not have a main graph, the graph creation unit 112 creates a main graph for the target combinatorial optimization problem, with each decision variable as a vertex. Edges in the main graph may be set between all vertices, or only between correlated decision variables.

[0045] Furthermore, in processing step S202, the graph creation unit 112 creates a subgraph with fewer vertices and edges than the main graph. In other words, the graph creation unit 112 creates a subgraph that is a reduced version of the main graph. Here, there only needs to be one or more subgraphs, and multiple subgraphs may be created. Clustering methods such as Louvain's method, KMeans method, or DBSCAN may be used to create the subgraphs. Alternatively, multiple vertices included in the same cluster may be compressed into a single vertex to create a subgraph, or each cluster may be treated as a separate subgraph.

[0046] Here, it is desirable for the graph creation unit 112 to store the main graph and the subgraphs created in processing step S202 as graph data D2 in the graph data storage unit DB2. The graph creation unit 112 also records in the graph data D2 which vertex of the subgraph corresponds to each vertex of the main graph.

[0047] Furthermore, in processing step S203, the mathematical representation creation unit 113 formulates a combinatorial optimization problem for each subgraph created in processing step S202. Here, the optimization solver selection unit 114 sets the optimization solver and / or algorithm to be used for solving. Then, the mathematical optimization unit 115 solves the combinatorial optimization problem on the subgraph. It is desirable that the mathematical optimization unit 115 stores the obtained solution as partial data D4 in the partial data storage unit DB4.

[0048] Furthermore, in processing step S204, the loss function creation unit 116 first refers to the partial data D4 and creates a loss function using the solution obtained in processing step S203 as training data. The machine learning unit 117 then creates a sub-GNN for each subgraph based on the information specified in the problem data D1. The machine learning unit 117 then trains the sub-GNN using the loss function stored in the loss function data D5. It is desirable that the machine learning unit 117 stores the feature vectors of each vertex in the first layer of the GNN obtained as a result of training as feature vector data D6 in the feature vector data storage unit DB6.

[0049] Furthermore, in processing step S205, the machine learning unit 117 assigns the feature vectors on the subgraph obtained in processing step S204 to the corresponding vertices of the original main graph. Here, the corresponding vertices of the main graph refer to the vertices that are the sources of each vertex in the subgraph.

[0050] Furthermore, in processing step S206, the loss function creation unit 116 creates a loss function for the main graph. Here, processing step S206 aims to solve a combinatorial optimization problem, and when creating the loss function, the loss function creation unit 116 refers to the mathematical representation data D3 to ensure that the learning result of the main GNN is the result of solving the combinatorial optimization problem. Next, the machine learning unit 117 creates a main GNN for the main graph based on the information specified in the problem data D1. Then, the machine learning unit 117 uses the feature vectors assigned in processing step S205 as input to the feature vectors of each vertex and trains the main GNN using the loss function stored in the loss function data D5.

[0051] In processing step S207, the solution output unit 119 reads the solution obtained in processing step S206, calculates the objective function value and the number of constraint violations, and outputs the solution and these calculation results. The solution output unit 119 outputs these to the output device 105 in Figure 1 and the communication device.

[0052] To briefly describe the optimization process shown in Figure 3 above, it involves three stages of computation: first, reducing the original main graph, for example, by compression and / or partitioning it to create smaller subgraphs; second, using the solutions obtained by a mathematical optimization solver for the combinatorial optimization problems on each subgraph as training data to train a sub-GNN; and third, using the resulting feature vectors as input to the main GNN to train the main GNN.

[0053] The following describes a specific example of the optimization process S200 using the maximum independent set problem as an example. As already mentioned, the maximum independent set problem is the problem of finding the largest independent set in a given graph G. For example, in a graph consisting of five vertices 1 to 5 as shown in Figure 4, one of the largest independent sets is {vertex 2, vertex 5}, and its size is 2. Below, keeping in mind the solution of the maximum independent set problem on a large graph, each processing step of the optimization process S200 will be described in detail step by step.

[0054] First, in processing step S201, the problem data acquisition unit 111 receives a specification from the user that the combinatorial optimization problem is a maximum independent set problem, and accepts input of the graph data to be solved. Here, the input graph is designated as the main graph. Other inputs such as settings for a graph neural network (GNN) used in later processing steps and selection conditions for a mathematical optimization solver may also be accepted, but the necessary inputs will be described later along with the explanation of the later processing steps.

[0055] Furthermore, in processing step S202, the graph creation unit 112 creates subgraphs smaller than the main graph by reducing the main graph through methods such as graph compression and graph partitioning. Known clustering methods such as Louvain's method, KMeans method, and DBSCAN may be used to create the subgraphs. The reduction methods may also include methods other than compression and partitioning, as well as combinations of two or more reduction methods, such as partitioning and compression.

[0056] Here, Figure 5 shows an example of subgraph creation using graph compression. In graph compression, the graph creation unit 112 creates a subgraph by treating each cluster as a single vertex. For example, if the two vertices shown in white in the main graph form one cluster, these vertices are combined into a single vertex in subgraph 1. Similarly, the graph creation unit 112 creates subgraphs for other clusters in the main graph by treating each cluster as a single vertex. Regarding the creation of edges in the subgraph, a clustering method may be followed, but for example, if edges exist between clusters in the main graph, edges may be created between the corresponding vertices in the subgraph.

[0057] Furthermore, the graph creation unit 112 may create multiple subgraphs. As shown in Figure 5, the graph creation unit 112 may further perform clustering on a subgraph 1 using a clustering method, compress the graph by considering each cluster as a vertex, and create subgraph 2. By repeating the same process, multiple subgraphs can be created, such as subgraph 3, subgraph 4, and so on, while making them smaller.

[0058] Next, Figure 6 shows an example of creating subgraphs using graph partitioning. When using graph partitioning, the graph creation unit 112 clusters the main graph and makes each cluster a separate subgraph. In Figure 6, the vertex groups of the diagonal pattern, the hollowed-out pattern, and the filled-in pattern are designated as subgraph 1, subgraph 2, and subgraph 3, respectively.

[0059] The subgraph created in processing step S201 must be of a size that can be adequately handled by the mathematical optimization solver used in the subsequent processing step S203. In other words, the role of processing step S201 is to reduce the size of the large main graph so that it can be handled by the mathematical optimization solver. As will be shown later in Figure 19, in order to preserve as much information as possible from the main graph, it is preferable that the size of the subgraph be as large as possible within the range that can be handled by the mathematical optimization solver.

[0060] Therefore, the graph creation unit 112 may use various parameter settings in the graph compression and graph partitioning algorithms specified in advance by the user in the problem data D1. That is, the graph creation unit 112 may create subgraphs and graphs using the parameter settings. Alternatively, the graph creation unit 112 may repeat the graph compression and graph partitioning algorithms while mechanically modifying the various parameter settings until a subgraph of the desired size is obtained. In other words, the graph creation unit 112 repeats the subgraph creation process while changing the setting parameters.

[0061] The various parameters in graph compression and graph partitioning algorithms include, for example, the resolution parameter which affects the size of the clusters and the number of clustering iterations in the Louvain method, and the number of clusters in the KMeans method.

[0062] The following explanation will use the case where a subgraph creation process is performed using the graph compression method shown in Figure 5 as an example. In processing step S203 in Figure 3, the mathematical representation creation unit 113 formulates a combinatorial optimization problem on the subgraph created in processing step S202 and performs a solution using a mathematical optimization solver. In this embodiment, the mathematical representation creation unit 113 performs a solution of the maximum independent set problem for each subgraph.

[0063] Here, Figure 7 shows a detailed flowchart of processing step S203, which is the subgraph creation process. First, in processing step S31, the mathematical representation creation unit 113 reads the subgraph created in processing step S202. Next, in processing step S32, the mathematical representation creation unit 113 formulates the maximum independent set problem for the read subgraph. For example, the maximum independent set problem can be formulated as shown in (Equation 1) below.

[0064]

number

[0065] Here, n, V, and E are the number of vertices, the set of vertices, and the set of edges of the subgraph, respectively, and x i is a binary variable that is 1 if vertex i belongs to an independent set, and 0 otherwise. P is a penalty coefficient that the user may specify in advance and store in the problem data D1. The first term of H in (Equation 1) corresponds to the size of the independent set, and the second term is a penalty term that prohibits adjacent vertices from both being included in an independent set.

[0066] Furthermore, in processing step S33, the mathematical representation creation unit 113 determines the characteristics of the mathematical representation of the combinatorial optimization problem specified in problem data 1 and the problem created in processing step S32, and selects the mathematical optimization solver to be used in the subsequent processing step S34. Here, Figure 8 shows a detailed flowchart of processing step S33 in Figure 7. First, in processing step S331, the mathematical representation creation unit 113 determines the characteristics of the given combinatorial optimization problem. Here, these characteristics include, for example, whether the objective function that serves as the optimization index in the mathematical representation created in processing step S32 is a quadratic or linear equation. It may also be determined whether the mathematical representation is described only by constraints, or whether there is a dedicated algorithm specific to the combinatorial optimization problem to be solved.

[0067] Next, in processing step S332, the mathematical representation creation unit 113 selects a mathematical optimization solver or algorithm to be used in subsequent processing based on the result of the feature determination in processing step S331. Here, general-purpose mathematical optimization solvers such as Ising machines, linear programming solvers, and constraint programming solvers may be selected. In addition, problem-specific algorithms such as greedy algorithms designed specifically for this problem may also be included as options. An example of a problem-specific algorithm for the maximum independent set problem is the Boppana-Halldorsson algorithm.

[0068] In processing step S332, the mathematical representation creation unit 113 selects a mathematical optimization solver according to the feature determination in processing step S331, as follows. For example, it may be selected as follows. If the objective function is determined to be a quadratic equation, then an Ising machine is used. If it is determined to be a linear equation, then a linear programming solver will be used. If it is determined that only constraints are present, then the constraint programming solver will be used. If a problem-specific algorithm is determined to exist, then it is the problem-specific algorithm.

[0069] However, the mathematical representation generation unit 113 may use a mathematical optimization solver specified in advance by the user in the problem data D1. Furthermore, when multiple conditions are met, such as the objective function being a quadratic expression and the algorithm being problem-specific, the choice of which mathematical optimization solver to use may be determined by specifying the priority of the mathematical optimization solvers. This priority may be stored in the problem data D1 in advance by the user.

[0070] In this embodiment, since the objective function is expressed as a quadratic equation as shown in (Equation 1), we will continue the explanation assuming that an Ising machine has been selected as the mathematical optimization solver.

[0071] Furthermore, in processing step S34 of Figure 7, the mathematical representation creation unit 113 solves the combinatorial optimization problem on the subgraph (in this embodiment, the maximum independent set problem) using the specified mathematical optimization solver. At this time, the mathematical representation created in processing step S32 may be used as input to the mathematical optimization solver. Next, in processing step S35, the mathematical representation creation unit 113 calculates the solution x obtained as a result of solving with the mathematical optimization solver. i Save (i=1, 2,…,N) as partial solution data D4 in Figure 2.

[0072] Furthermore, in processing step S36, the mathematical representation creation unit 113 determines whether to terminate the mathematical optimization solver process for the subgraph. The determination conditions include whether the solving process has been executed for all created subgraphs, and whether the processing time specified by the user in the problem data D1 has been reached. If it is determined that the solving process should be terminated (Yes), processing S203, which is the subgraph creation process, is terminated. If it is not determined that the solving process should be terminated (No), the process returns to processing step S31, the data for the next subgraph is acquired, and processing steps S32 to S36 are repeated. However, in Figure 7, the mathematical representation creation unit 113 executes the solving process sequentially for each subgraph, but it is also acceptable to execute the solving process for multiple subgraphs in parallel.

[0073] Returning to FIG. 3 again, the procedure of the optimization process will be described. In processing step S204, the machine learning unit 117 creates a GNN for each sub-graph, uses the solution obtained by solving with a mathematical optimization solver as teacher data, and performs learning of the GNN.

[0074] A GNN is a neural network for data with a graph structure, which propagates and processes information on the graph. Also, each vertex on the graph has numerical data on a vector called a feature vector. In a GNN, the feature vector is updated by transmitting information between adjacent vertices. Let the feature vector of vertex i in the k-th layer of the GNN be f i (v) Then, the feature vector of the (k + 1)-th layer is determined, for example, as shown in the following (Equation 2).

[0075]

Equation

[0076] Here, R (k) , N(i) are the activation function of the k-th layer and the set of vertices adjacent to vertex i, respectively. W (k) , B (k) are the weights for information propagation between vertices in the k-th layer and are learning parameters in machine learning.

[0077] Also, the above GNN is schematically shown in FIG. 9. In FIG. 9, the GNN is composed of three layers. And the feature vectors of each layer are f1 (1) ~f3 (1) are shown. Here, as the GNN, a graph convolutional network (GCN) is taken as an example, but other types of GNNs such as a graph attention network (GAT) may also be used. Note that it is desirable for the user to store in advance in the problem data D1 settings related to the structure of the GNN, such as whether it is GCN or GAT, the dimension of the feature vector, the number of layers of the GNN, the activation function, and the initial value of the learning rate. However, the dimension of the feature vector in the final layer of the GNN should be the same as the variable of the combinatorial optimization problem assigned to each vertex of the sub-graph. In the case of the maximum independent set problem, the variable x of each vertexi Since is a binary variable indicating whether or not a value is included in an independent set and is a scalar (one-dimensional), the feature vector of the final layer of the GNN is also set to be a scalar (one-dimensional).

[0078] Furthermore, in GNN training, a loss function is defined using the feature vector of the final layer, and W is used to minimize that loss function. (k) B (k) The learning parameters are optimized as follows. In this embodiment, Adaptive Moment Estimation (Adam) is used as the algorithm for optimizing the learning parameters, but other methods such as stochastic gradient descent or Adaptive Gradient Algorithm (AdaGrad) may also be used.

[0079] Next, we will explain the details of processing step S204 in Figure 3. Figure 10 is a flowchart detailing processing step S204 in Figure 3, which is the sub-GNN learning process. First, in processing step S41, the loss function creation unit 116 acquires the data of the subgraph to be learned. Then, in processing step S42, the loss function creation unit 116 acquires the solution of the mathematical optimization solver for the target subgraph from the partial data D4. Then, in processing step S43, the loss function creation unit 116 creates a loss function from the acquired solution of the mathematical optimization solver and the mathematical representation of the combinatorial optimization problem created in processing step S32 in Figure 7.

[0080] Here, a detailed flowchart of processing steps S42 and S43 is shown in Figure 11. For convenience, Figure 11 (processing steps S42 and S43) will be referred to as the loss function creation process S4243 for the sub-GNN. In processing step S42431 of the loss function creation process S4243 for the sub-GNN, the loss function creation unit 116 obtains the solution of the mathematical optimization solver, and x sol This corresponds to processing step S42 in Figure 10. In processing steps S42-S432, the loss function creation unit 116 creates the loss function L(x) for the GNN (sub-GNN) for the target subgraph. Here, the solution x of the mathematical optimization solver is solThe loss function is the sum of the squared error and the objective function of the combinatorial optimization problem (function H in (Equation 1)). However, the weight coefficients a and b set for each term are specified in advance by the user for the problem data D1. Also, one of these weight coefficients may be 0.

[0081] Note that the loss function is x sol The squared error is set to allow the GNN to mimic the solution of the mathematical optimization solver. In processing step S42433, the loss function creation unit 116 saves the loss function created for the sub-GNN as loss function data D5. Processing steps S42432 and S42433 correspond to processing step S43 in Figure 10. With this, in processing step S42434, the creation of the loss function for the sub-GNN is completed.

[0082] Here, we will explain the learning process of the sub-GNN using Figure 12. Figure 12 shows the results of solving the maximum independent set problem with two different loss functions, treating instance C125-9 of the DIMACS benchmark set as a subgraph. In Figure 12, the vertical axis is the objective function H (Equation 1), and the horizontal axis is the computation time.

[0083] In the GNN-only example in Figure 12, the weight coefficient a=0, and the objective function is used directly as the loss function. In this example, the solution x of the mathematical optimization solver sol It does not receive any information and will create the subgraph using only the GNN. On the other hand, in GNN&IM, the solution x of the mathematical optimization solver is obtained by setting the weight coefficient b=0. sol The loss function is the squared error of the given expression. In this embodiment, an Ising machine was used as the mathematical optimization solver. In Figure 12, the time required to solve the problem using the Ising machine is also included in the display.

[0084] Comparing the results of GNN only and GNN&IM, we can see that GNN&IM learned faster and with higher accuracy. This indicates that the learning performance of a GNN can be improved by obtaining information from the solution of an Ising machine, rather than performing the solution on its own. However, this is the case when learning is performed on a graph of a size that can be handled by a mathematical optimization solver. When dealing with a huge main graph, the information obtained by the mathematical optimization solver will be transmitted to the main GNN via a sub-GNN.

[0085] Returning to Figure 10, let's explain the processing flow. In processing step S44, the machine learning unit 117 constructs a sub-GNN with the settings specified by the user in the problem data D1, and trains the GNN using the loss function created in processing step S43. The learning parameters in the sub-GNN and the initial values ​​of the first-layer feature vectors may be determined by the machine learning unit 117 either by receiving specifications from the user in the problem data D1 or by random determination. The user may also specify the termination conditions for the loss function. Examples of termination conditions include setting upper limits on computation time and the number of epochs, or until no improvement in the loss function value beyond a specified value is observed within a specified number of epochs.

[0086] In processing step S45, the machine learning unit 117 saves the first-layer feature vector of the sub-GNN obtained as a result of learning as feature vector data D6. In processing step S46, the machine learning unit 117 determines whether to terminate processing step S204, which is the sub-GNN learning process. The conditions for this determination are the same as those for processing step S36 in Figure 7, and include whether learning has been performed for all subgraphs and whether the processing time specified by the user has been reached. If the sub-GNN learning is to be continued (No), the process returns to processing step S41 to obtain data for another subgraph, and processing steps S42 to S46 are repeated. However, although the sub-GNN learning process is performed sequentially for each subgraph in Figure 10, the sub-GNN learning process may be performed in parallel for multiple subgraphs.

[0087] Let's return to Figure 3 for further explanation. In processing step S205, the feature vector assignment unit 118 assigns the feature vectors obtained from training each sub-GNN to each vertex of the main GNN (main GNN) to be used as initial values ​​(feature vectors of the first layer) for the GNN (main GNN) relative to the main graph.

[0088] This assignment will be explained using Figure 13. Figure 13 shows the method of assigning feature vectors in processing step S205 of Figure 3. In the following, the vertex shown as a hollow in the main graph will be referred to as vertex 1, and the first layer feature vector at this vertex 1 will be f 1,main (1) Let's assume that... Hereafter, f 1,main (1) I will explain how to make that decision.

[0089] First, in the graph compression step S202 of Figure 3, let's assume that the cluster containing vertex 1 of the main graph is compressed to a hollow point in subgraph 1. Let's call this hollow point in subgraph 1 vertex 1' of subgraph 1. In this case, vertex 1 of the main graph corresponds to vertex 1' in the subgraph. Furthermore, if the cluster containing vertex 1' of subgraph 1 is compressed to a hollow vertex 1'' of subgraph 2, then vertex 1 of the main graph corresponds to vertex 1'' in subgraph 2. The graph creation unit 112 can then find the vertex corresponding to vertex 1 of the main graph for each subgraph in the same manner.

[0090] Furthermore, as shown in the lower part of Figure 13, for each subgraph vertex corresponding to vertex 1 of the main graph, the average value of their feature vectors is f 1,main (1)This may be done. In the figure, N represents the number of subgraphs. Here, the feature vectors at the vertices of the subgraphs refer to the feature vectors obtained during learning in processing step S204 in Figure 3, and can be read from the feature vector data D6 in Figure 2. In this embodiment, the average value of the feature vectors of all subgraphs is simply used, but weights may be assigned to the feature vectors of each subgraph. These weights may be specified in advance by the user, or they may be determined according to the number of nodes in the subgraph.

[0091] The method for determining the initial value of the feature vector was explained using vertex 1 of the main graph as an example, but the same process is performed for all vertices of the main graph. In this embodiment, the feature vector assignment unit 118 determines which vertex of the main graph each vertex of the subgraph was created from, and sets a linear combination of the feature vectors obtained during the training of the sub-GNN as the initial input value of the feature vector for each vertex of the main GNN, for the group of subgraph vertices corresponding to each vertex of the main GNN. Through this process, the information obtained during the training of the sub-GNN can be passed on to the main GNN. Furthermore, since the sub-GNN obtains information from the mathematical optimization solver, the main GNN can consequently receive information from the mathematical optimization solver.

[0092] Returning to Figure 3, let's explain the processing flow. In processing step S206, the machine learning unit 117 solves the combinatorial optimization problem that was originally intended to be solved, that is, the combinatorial optimization problem on the original main graph, by training the main GNN. Similar to the sub-GNN, the configuration conditions of the GNN can be specified in advance by the user using the problem data D1. However, in solving the maximum independent sets problem, the feature vector of the final layer is still one-dimensional (scalar).

[0093] In this embodiment, the mathematical representation of a combination optimization problem is used as the loss function for training the main GNN. In the case of the maximum independent set problem, the mathematical representation (Equation 1) created in processing step S32 of Figure 7 can be used. Note that, as shown in the second term of (Equation 1), constraints may be included in the objective function H (= loss function L) as penalty terms. When creating the loss function, the variable x i If x is a binary variable of 0 / 1, then we can use the relationship shown in (Equation 3) to determine x i All linear equations can be converted into quadratic equations.

[0094]

number

[0095] Therefore, the loss function L can be expressed solely as a quadratic equation, as shown in (Equation 4).

[0096]

number

[0097] Furthermore, if we transform (Equation 1), which is the objective function of the largest independent set, we get (Equation 5).

[0098]

number

[0099] Also, Figure 14 shows the representation of (Equation 4) x T Qx is shown in matrix form, and the diagonal terms of the interaction matrix Q have finite values. T Differentiating the loss function in Qx form with respect to a learning parameter p yields (Equation 6).

[0100]

number

[0101] (Equation 6) is x=0 (for any i xi Since it becomes 0 at x=0, the loss function in (Equation 4) has a local minimum at x=0. Therefore, using the loss function in (Equation 4) tends to easily trap the local minimum at x=0 during GNN training.

[0102] Therefore, as in (Equation 1), it is best to separate the linear expression in variable x from the quadratic expression and treat it as a linear expression. In other words, the loss function L should be in the form of (Equation 7).

[0103]

number

[0104] Here, h in (number 7) i (Equation 4) is the coefficient q when i=j i,i It is equal to the coefficient q in (Equation 7). i,j and h i When we match this to equation (Equation 1), we get (Equation 8) and (Equation 9).

[0105]

number

[0106]

number

[0107] Here, we will explain an example of separating the loss function of the main GNN into a quadratic and a linear equation using Figure 15. Figure 15 shows the representation of (Equation 7) x T Qx+h T x is shown in matrix form, and all the diagonal terms of the interaction matrix Q are zero. Therefore, differentiating (Equation 7) with respect to a learning parameter p yields (Equation 10).

[0108]

number

[0109] Unlike (Equation 6), (Equation 10) does not become 0 even at x=0, so x=0 is not a local minimum in the loss function of (Equation 7). Therefore, learning using the loss function of (Equation 7) tends to proceed more smoothly than using (Equation 4). For this reason, it is desirable to use (Equation 10) preferentially in this embodiment. Here, using preferentially means using (Equation 10), or if the learning result using (Equation 10) is not the desired result, (Equation 4) may be used. Alternatively, both may be used.

[0110] Here, we will explain the comparison of the learning processes of the main GNN with different loss functions using Figure 16. Figure 16 shows the GNN solution process for the maximum independent set problem for instance frb40-19-1 of the BHOSLIB benchmark set. In Figure 16, the vertical axis is the loss function value and the horizontal axis is the learning time. The penalty coefficient P in the loss function is set to 2, and x (Equation 4) T Qx format and (Math 7) x T Qx+h T We performed training using a two-layer GCN with two different loss functions in x form.

[0111] As a result, x T In the Qx format, the learning process stalled when it got trapped in a local minimum at x=0, with the loss function value remaining at 0, whereas x T Qx+h T x format is x T We confirmed that training progressed to a lower (better) loss function value than that achieved with the Qx format. This trend was observed in other instances of the BHOSLIB benchmark set, and can also be confirmed in the DIMACS benchmark set.

[0112] In processing step S206, the output of the main GNN training is the feature vector (a scalar in the case of the maximum independent sets problem) for each vertex of the final layer. While it is desirable to obtain the binary variable value of each vertex as the solution to the maximum independent sets problem, the values ​​obtained through the activation function in a GNN are not necessarily binary variables, but are generally obtained as continuous values.

[0113] Next, using Figure 17, we will explain the procedure for rounding the continuous output obtained as a result of learning the main graph to a binary value. Figure 17 shows how the continuous values ​​obtained from GNN learning are converted to binary variables. For example, when the sigmoid function is used as the activation function, the output x of the GNN i x is a continuous value contained in the interval [0,1] (see the "Learning" process in the figure). Therefore, it is necessary to convert these continuous values ​​into binary variables based on some criterion. In the example in Figure 17, x i If the value is 0.5 or greater, it is rounded to 1; if it is less than 0.5, it is rounded to 0 (see the "rounding process" in the diagram). As a result, the variables assigned to vertices 2 and 5 are converted to 1, and the variables assigned to vertices 1, 3, and 4 are converted to 0. Such rounding processes do not necessarily have to be judged based on the magnitude of a clear reference value (0.5 in this case); they could also be performed by probabilistically rounding to 0 or 1, reflecting the continuous values ​​obtained as a result of GNN learning.

[0114] In processing step S207, the solution output unit 119 uses the binary variable solution x obtained in processing step S206 to calculate the objective function value and the number of constraint violations, which serve as optimization indicators, and outputs these calculation results along with the solution x. At this time, the continuous value of x before rounding may also be output as part of the GNN output. In the case of the maximum independent set problem, for example, in addition to the objective function value of (Equation 1), the size of the independent set (the first term of (Equation 1)H) and the number of constraint violations, x, are also output. i The number of edges between vertices where the value is 1 (the second term of equation H in equation 1) is calculated. This result is presented to the user via output device 105 or communication device 106.

[0115] The optimization process has been explained above following the flowchart shown in Figure 3. Finally, we will discuss the performance of this optimization process. Here, we will explain the impact of the optimization process on the solution performance. Figure 18 is a table comparing the performance of each solution method for the maximum independent set problem on an artificially created graph with 100,000 vertices and all vertices having a degree of 5. In Figure 18, the loss function value is calculated using the continuous values ​​within the [0,1] interval output by the main GNN before performing the rounding process in Figure 17, where H in (Equation 1) is used. However, the penalty coefficient P was set to 2. The number of constraint violations is calculated in processing step S207 in Figure 3 after determining which vertices are included in the independent set by rounding the variables of each vertex to 0 or 1.

[0116] In this embodiment, when solving using only the main GNN, the loss function is created directly from the main graph without creating subgraphs, the feature vectors of the main GNN are initialized randomly, and the main GNN is trained. This is equivalent to omitting processing steps S202 to S206 in the optimization process S200.

[0117] Furthermore, in the case of the main GNN & sub-GNN, the combinatorial optimization problem on each subgraph is solved by the sub-GNN alone. This can be achieved by setting the weight coefficient a=0 in (equation 11) of processing step S42432.

[0118]

number

[0119] This optimization process is essentially equivalent to skipping processing step S203 and training the sub-GNN without training data in processing step S204.

[0120] In the case of the main GNN, sub-GNN, and Ising machine, all processing steps shown in Figure 3 are executed, and in this embodiment, an Ising machine was used as the mathematical optimization solver. In each solving method, the GNN configuration is a two-layer GCN, with the ReLU function used as the activation function for the first layer and the sigmoid function for the second layer. The Louvain method was used to create the subgraphs in both the main GNN & sub-GNN and the main GNN & sub-GNN & Ising machine.

[0121] Furthermore, comparing the results of the main GNN and the main GNN & sub-GNN, it can be seen that the loss function value is ultimately improved by utilizing the results obtained by compressing the huge main graph into subgraphs and solving them. In addition, comparing the results of the main GNN & sub-GNN and the main GNN & sub-GNN $ Ising machine, it can be seen that the loss function value is further improved by utilizing the Ising machine solution for the subgraphs. Moreover, only the main GNN & sub-GNN & Ising machine is able to output a feasible solution with zero constraint violations, confirming the effectiveness of executing all the processing steps in Figure 3.

[0122] Next, we will discuss the relationship between the size of the subgraph and the solution accuracy of the main GNN. When compressing or partitioning the main graph to create subgraphs, multiple subgraphs may be created as long as they are small enough to be handled by a mathematical optimization solver. However, in extremely small subgraphs compared to the main graph, most of the features and similarities of the main graph are lost, so it is expected that the effect of transferring information obtained from training the sub-GNN to the main GNN will be small.

[0123] Therefore, we created an artificial main graph with 150,000 vertices and a degree of 5 for all vertices, and investigated how the output of the main GNN changes depending on which subgraph's training results, compressed using the Louvain method, are used. The Ising machine used in this example can handle up to 100,000 variables. Therefore, we created subgraphs so that their size is 100,000 vertices or less.

[0124] Here, we will explain the relationship between the number of subgraphs used for learning in this embodiment and the solution accuracy of the main GNN. The table in Figure 19 shows the performance difference from using up to subgraph 1 to using subgraph 4. The number of vertices refers to the number of vertices of the smallest graph used as a subgraph. That is, the number of vertices for "up to subgraph 1" refers to the number of vertices of subgraph 1, and the number of vertices for "up to subgraph 2" refers to the number of vertices of subgraph 2. The degree to which the loss function value is achieved shows the ratio of the loss function value of the main GNN to that of the case where up to subgraph 7 is used.

[0125] As shown in Figure 19, the loss function value reaches 94.06% when the learning results of the largest subgraph, 1, are used, and it can be seen that the improvement in the loss function value slows down when the results of smaller subgraphs are incorporated thereafter. This means that most of the useful information acquired by the main GNN is passed on from subgraph 1. In other words, it is important to use the solution results of subgraphs that are as close in size to the main graph as possible in order to improve the solution accuracy. Considering that the sub-GNN is assisted by the mathematical optimization solver in solving subgraphs, it can be said that creating the largest possible subgraph within the range that the mathematical optimization solver can handle is important in order to achieve higher solution accuracy. Thus, in this embodiment, "close" in solving refers to the closeness in terms of graph size.

[0126] Although one embodiment of the present invention has been described in detail above, it goes without saying that the present invention is not limited to the above-described embodiment and can be modified in various ways without departing from its essence. For example, the above-described embodiment has been described in detail in order to explain the present invention in an easy-to-understand manner and is not necessarily limited to having all the configurations described. Furthermore, it is possible to add, delete, or replace some of the configurations of the above embodiment with other configurations.

[0127] Furthermore, each of the above-mentioned configurations, functional units, processing units, processing means, etc., may be implemented in hardware, either partially or entirely, for example, by designing them as integrated circuits. Alternatively, each of the above-mentioned configurations, functions, etc., may be implemented in software by having the processor interpret and execute programs that realize each function. Information such as programs, tables, and files that realize each function can be stored in memory, hard disks, SSDs (Solid State Drives), or other recording devices, or in recording media such as IC cards, SD cards, or DVDs.

[0128] Furthermore, in the diagrams above, the control lines and information lines shown are those deemed necessary for explanation and do not necessarily represent all control lines and information lines that would be present in the actual implementation. For example, it can be assumed that almost all components are interconnected in practice.

[0129] Furthermore, the arrangement of the various functional units, processing units, and databases of the information processing device 100 described above is merely an example. The arrangement of the various functional units, processing units, and databases can be changed to the optimal arrangement from the perspective of the performance, processing efficiency, and communication efficiency of the hardware and software provided by the information processing device 100.

[0130] Furthermore, the configuration of the database (schema, etc.) used to store the various types of data mentioned above can be flexibly modified from the perspective of efficient resource utilization, improved processing efficiency, improved access efficiency, and improved search efficiency. [Explanation of Symbols]

[0131] 100: Information Processing Device 107: Combinatorial Optimization Device 108: Machine Learning Device 111: Problem Data Acquisition Unit 112: Graph Creation Department 113: Mathematical Representation Creation Department 114: Optimization solver selection unit 115: Mathematical Optimization Department 116: Loss Function Creation Section 117: Machine Learning Department 118: Feature vector assignment section DB: Storage section DB1: Problem data storage unit DB2: Graph Data Storage Unit DB3: Mathematical Representation Data Storage Unit DB4: Partial Data Storage Unit DB5: Loss function data storage unit DB6: Feature vector data storage unit

Claims

1. An information processing device for processing combinatorial optimization problems that can be defined on a graph, A problem data acquisition unit into which problem data is input, A graph creation unit that, referring to the aforementioned problem data, creates one or more subgraphs by reducing the main graph, which is the graph to be solved, A mathematical optimization unit that solves the combinatorial optimization problem for each of the aforementioned subgraphs using a mathematical optimization solver, A machine learning unit trains each of the sub-GNNs using the solutions obtained by the mathematical optimization solver as training data, so that the output of the sub-GNN corresponding to each subgraph is close to the solution of the mathematical optimization solver. A feature vector assignment unit assigns the feature vectors at each vertex of the sub-GNN obtained as a result of learning to the corresponding vertices of the main GNN, using them as input to the feature vectors of the main GNN corresponding to the graph data of the main graph. An information processing device having a solution output unit that outputs a solution obtained as a result of training a main GNN by setting a loss function so as to solve a combinatorial optimization problem for the main graph in the machine learning unit.

2. An information processing apparatus according to claim 1, Furthermore, when the objective function of the combinatorial optimization problem includes a quadratic expression, the information processing device has a mathematical representation creation unit that converts the quadratic terms, which are expressed by the square of the same binary variable, into linear terms while maintaining the coefficients of the quadratic terms, and creates a mathematical representation of the objective function using a quadratic expression consisting only of intersection terms, which are products of different variables, and a linear expression including the converted linear terms.

3. An information processing apparatus according to claim 1, The graph creation unit is an information processing device that reduces the size of the main graph to create one or more subgraphs that are smaller in scale than the main graph.

4. An information processing apparatus according to claim 3, The graph generation unit divides the main graph into clusters using a predetermined clustering method. The subgraph is created by graph compression, which treats each cluster divided into the aforementioned clusters as a single vertex to create a subgraph, and / or graph partitioning, which treats each cluster divided into the aforementioned clusters as a separate subgraph. An information processing device characterized by the following:

5. An information processing apparatus according to claim 4, The graph creation unit is an information processing device that obtains the largest subgraph by repeatedly performing a subgraph creation process while changing the setting parameters in the clustering method.

6. An information processing apparatus according to claim 1, Furthermore, the information processing device determines the characteristics of the combinatorial optimization problem and, according to the determined characteristics, selects a mathematical optimization solver: an Ising machine if the objective function is determined to be quadratic, a linear programming solver if it is linear, a constraint programming solver if there are only constraints, and a problem-specific algorithm if there is one.

7. An information processing apparatus according to claim 1, Furthermore, the information processing device has a loss function creation unit that creates a loss function as a mathematical representation obtained by multiplying the squared error with the solution of the subgraph and the objective function of the combinatorial optimization problem defined on the subgraph by their respective weights and adding them together.

8. An information processing apparatus according to claim 4, The graph creation unit determines which vertex of the main graph each vertex of the subgraph was created from, and sets a linear combination of feature vectors obtained during the training of the sub-GNN as the initial input value for the feature vector of each vertex of the main GNN, for the group of vertices of the subgraph corresponding to each vertex of the main GNN.

9. An information processing method executed by an information processing device that processes combinatorial optimization problems that can be defined on a graph, The problem data acquisition unit receives the input of problem data, When the graph creation unit refers to the problem data and creates one or more subgraphs by reducing the size of the main graph, which is the graph to be solved, The mathematical optimization unit solves the combinatorial optimization problem for each of the subgraphs using a mathematical optimization solver. The machine learning unit trains each of the sub-GNNs corresponding to each subgraph, using the solutions obtained by the mathematical optimization solver as training data, so that the output of the sub-GNN is close to the solution of the mathematical optimization solver. The feature vector assignment unit assigns the feature vectors at each vertex of the sub-GNN obtained as a result of learning to each vertex of the corresponding main GNN, using them as input to the feature vectors of the main GNN corresponding to the graph data of the main graph. An information processing method in which a solution output unit outputs a solution obtained as a result of the machine learning unit training a main GNN by setting a loss function to solve a combinatorial optimization problem on the main graph.

10. The information processing method according to claim 9, An information processing method in which the graph creation unit reduces the size of the main graph to create one or more subgraphs that are smaller in scale than the main graph.

Images