Machine learning program, determination program, machine learning method, determination method, machine learning device, and determination device

Abstract

This machine learning program causes a computer to perform a process for machine-learning a model by using a loss term that matches the degree of continuity and discreteness of a variable to be optimized, as a cost function in a search process that performs a search with continuous relaxation incorporated into a discrete optimization problem, and by changing the loss term as the search process progresses.　

Description

Machine learning program, judgment program, machine learning method, judgment method, machine learning device, and judgment device

[0001] The present invention relates to a machine learning program, a determination program, a machine learning method, a determination method, a machine learning device, and a determination device.

[0002] Techniques for optimizing complex combinations have been disclosed (see, for example, Non-Patent Documents 1 and 2).

[0003] Schuetz, MJ, Brubaker, JK, and Katzgraber, HG (2022a). Combinatorial optimization with physics-inspired graph neural networks. Nature Machine Intelligence, 4(4):367-377. Schuetz, MJ, Brubaker, JK, Zhu, Z., and Katzgraber, HG (2022b). Graph coloring with physics-inspired graph neural networks. Physical Review Research, 4(4):043131.

[0004] In combinatorial optimization, a continuous relaxation method using a machine learning model is considered to search for an optimal solution. However, in the continuous relaxation method, the solution obtained may differ from the original solution. This may result in a decrease in the accuracy of the solution of the combinatorial optimization problem.

[0005] In one aspect, the present invention aims to provide a machine learning program, a judgment program, a machine learning method, a judgment method, a machine learning device, and a judgment device that can improve the accuracy of solving combinatorial optimization problems.

[0006] In one aspect, the machine learning program causes a computer to execute a process of machine learning a model by using a loss term according to the degree of continuity and discreteness of the variable to be optimized as a cost function in a search process that incorporates continuous relaxation into a discrete optimization problem, and by changing the loss term as the search process progresses.

[0007] In another aspect, the determination program causes the computer to execute a process of embedding an optimization problem in the model obtained by executing a machine learning program to output a solution.

[0008] The accuracy of solving combinatorial optimization problems can be improved.

[0009] 1 is a diagram illustrating conversion to a graph embedding vector. FIG. 2 is a diagram illustrating the results of a Maximum Independent Set on a 50-Regular Random Graph. FIG. 3 is a diagram illustrating the results of application to an MIS. (a) and (b) are diagrams illustrating the results of application to an MIS. (a) is a functional block diagram illustrating the overall configuration of an information processing device according to a first embodiment, and (b) is a hardware configuration diagram of the information processing device. FIG. 4 is a flowchart illustrating an example of the operation of an information processing device during machine learning. FIG. 5 is a flowchart illustrating an example of the operation of an information processing device when optimization is performed using the results of machine learning.

[0010] Optimization problems exist in a variety of industries, including manufacturing and distribution. In particular, combinatorial optimization problems, which involve optimizing combinations, are one of the most important areas in the field of optimization. Combinatorial optimization problems are applied in a variety of fields, including transportation, logistics, communications, and finance.

[0011] First, we will explain the continuous relaxation method using a machine learning model. The continuous relaxation method is a method in which, instead of solving a discrete optimization problem, the discrete optimization problem is relaxed and a continuous optimization problem corresponding to the discrete optimization problem is solved.

[0012] As an example, a continuous relaxation solution method in the QUBO format will be described. QUBO stands for Quadratic Unconstrained Binary Optimization, which is a format that allows binary optimization without quadratic constraints.

[0013] First, optimization problems in the QUBO format can be expressed as a loss function as shown in the following formula (1). Problems such as the traveling salesman problem, the knapsack problem, and the graph coloring problem can be mapped in the QUBO format. x is a vector represented by 0 and 1, and has N elements. x Ｔ is the transpose matrix of x.

[0014] In the continuous relaxation solving method, the QUBO of the above formula (1) is relaxed to some simple form. As an example, the above formula (1) is relaxed to a hypercubic lattice. For example, the above formula (1) is expressed as the following formula (2). In the following formula, [0, 1] Ｎ represents an N-dimensional hypercubic lattice that takes on values ​​of 0 or 1.

[0015] However, even if QUBO is continuously relaxed, the Loss Landscape may still become complicated. Also, the optimal solution after the relaxation may differ significantly from the original optimal solution.

[0016] Next, it is considered that p in the above formula (2) is parameterized by a GNN (Graph Neural Network) and optimized using the loss function of the following formula (3). In the following formula (3), G of the optimization problem is expressed as an embedding vector h Ｇ，φ G is a feature vector of the graph in the GNN. For example, in FIG. 1, the feature vector G of the graph is converted into a graph embedding vector with the number of nodes N=4 and the number of edges E=4. For example, the optimization problem of p is reduced to the optimization of (θ, φ) in the above formula (3) characterized by the parameters of the GNN. In the following formula (3), R Ｅ×Ｎ is an E×N-dimensional Euclidean space.

[0017] However, the continuous relaxation solution method is not good at solving combinatorial optimization problems on graphs with high degrees (the number of edges connected to each node). For example, depending on the content of the optimization problem, the search for the optimal solution may not progress and may converge to a trivial discrete local solution p = (0, ..., 0) at an early stage. Figure 2 shows an example of failure. Figure 2 shows the results of a maximum independent set on a 50-regular random graph. The dotted line represents the cost function. The solid line represents the observable that measures the discrete-continuous difference. In the example in Figure 2, it can be seen that a trivial discrete local solution is reached in the early stages of machine learning, and the search no longer progresses.

[0018] Therefore, in the following embodiment, an example will be described in which the accuracy of the model can be improved and the accuracy of solving combinatorial optimization problems can be improved.

[0019] First, the principle of this embodiment will be described.

[0020] A discrete vector p∈{0,...,p} with q values Ｎ Discrete optimization min on ｐ A penalty term R(p) is introduced as a cost function for H(p) as shown in the following equation (5). The penalty term R(p) is a loss term for controlling the degree of continuity and discreteness. I is a parameter that characterizes the nature of the problem. The method of this embodiment is not only applicable to the QUBO format, but also to other formats. As an example, in the QUBO format, the QUBO matrix Q is a parameter that characterizes the problem.

[0021] λ is a parameter for controlling the penalty term, and is a hyperparameter for controlling the degree of continuity and discreteness. For example, if λ<0, a continuous solution is preferred, and if λ>0, a discrete solution is preferred.

[0022] In this embodiment, as an example, as the machine learning progresses, the hyperparameter λ is changed to a negative value λ （０） < 0 to positive value λ （Ｔ）> 0. As a result, as machine learning progresses, the penalty term changes from one in which the loss decreases as the discrete vector p becomes more continuous to one in which the loss increases as the discrete vector p becomes more continuous. λ (0) to (T) represent, for example, time, with one step of learning θ and φ being the unit time. In this embodiment, this technique of gradually changing the penalty term as machine learning progresses is referred to as continuous relaxation annealing.

[0023] A continuous space is relatively easy to optimize and allows for diverse exploration. Therefore, it is preferable to perform diverse exploration in a continuous space in the early stages of machine learning, for example by adjusting the penalty term, and to map continuous solutions naturally to discrete solutions in the later stages of machine learning.

[0024] For example, when the continuous relaxation annealing according to this embodiment is applied to a solution-finding method using a GNN, it can be regarded as optimization with respect to θ and φ as shown in the following equation (6).

[0025] The annealing method for the hyperparameter λ is arbitrary. Therefore, annealing may be performed in any manner, such as exponentially or linearly. For example, the hyperparameter λ may be gradually annealed using linear scheduling as shown in the following equation (7). Note that t represents the time in which one step of machine learning of θ and φ is taken as the unit time. ε is a small positive constant.

[0026] The continuous relaxation annealing of this example was applied to the MIS (Maximal Independent Set). The MIS is a problem to find the largest independent set on a graph. An independent set is a set of nodes such that both edges of the set are included when any other vertex is added. The results of continuous relaxation annealing applied to the MIS problem with 10,000 nodes, with λ = 0.0 and λ = -3.0, are described below.

[0027] FIG. 3 shows the degree dependency (d dependency) of the size of the independent set. The vertical axis represents the size of the independent set. The larger the value on the vertical axis, the better the approximate solution obtained. If λ = 0.0, as the degree of the graph increases, a trivial 0-bit solution is reached and the independent set cannot be searched for. On the other hand, in this embodiment, a good approximate solution is obtained even when d is large. In FIG. 3, "Annealing" shows the results of this embodiment. FIG. 4(a) shows the d dependency of the cost function. FIG. 4(b) shows the d dependency of the penalty term.

[0028] This embodiment can also be applied to a discrete variable pε{1, ..., q} (Potts variable) having q values. For example, p=(3, 2, q, ..., q−1) is converted into a one-hot matrix as shown in the following equation (8).

[0029] In this case, the degree of continuity and discreteness can be controlled by using the penalty term R(p) of the following equation (9) for the one-hot vector.

[0030] Next, a device configuration for realizing the above-described solution principle will be described. Fig. 5(a) is a functional block diagram showing the overall configuration of an information processing device 100 according to the first embodiment. The information processing device 100 is, for example, a server for optimization processing. As illustrated in Fig. 5(a), the information processing device 100 functions as an optimization problem storage unit 10, a model parameter storage unit 20, a node embedding unit 30, a relaxation variable unit 40, a loss function calculation unit 50, a gradient storage unit 60, an approximate solution output unit 70, and the like. The information processing device 100 functions as a machine learning device during machine learning and as a determination device during determination.

[0031] Fig. 5B is a hardware configuration diagram of the information processing device 100. As illustrated in Fig. 5B, the information processing device 100 includes a CPU 101, a RAM 102, a storage device 103, an input device 104, a display device 105, and the like.

[0032] The CPU (Central Processing Unit) 101 is a central processing unit. The CPU 101 includes one or more cores. The RAM (Random Access Memory) 102 is a volatile memory that temporarily stores programs executed by the CPU 101, data processed by the CPU 101, and the like. The storage device 103 is a non-volatile storage device. Examples of the storage device 103 include a read-only memory (ROM), a solid-state drive (SSD) such as a flash memory, and a hard disk driven by a hard disk drive. The storage device 103 stores a machine learning program and a judgment program. The input device 104 is a device for a user to input necessary information, such as a keyboard or a mouse. The display device 105 is a display device that displays the approximate solution output by the approximate solution output unit 70 on a screen. The CPU 101 executes the calculation program or the machine learning program to realize each unit of the information processing device 100. Each unit of the information processing device 100 may be implemented by hardware such as a dedicated circuit.

[0033] 6 is a flowchart illustrating an example of the operation of the information processing device 100 during machine learning. As illustrated in FIG. 6, the loss function calculation unit 50 initializes the model (step S1). Specifically, the loss function calculation unit 50 sets the model parameters stored in the model parameter storage unit 20 to predetermined initial values.

[0034] Next, the node embedding unit 30 embeds the optimization problem (step S2). For example, in a problem using a graph, the node embedding unit 30 embeds the graph feature vector of the given optimization problem into an embedding vector h φ，Ｇ The relaxation variable unit 40 sets the dynamic variables to be relaxed and parameterized by the neural network, thereby obtaining the loss function expressed by the above formula (6).

[0035] Next, the loss function calculation unit 50 updates the model parameters by the gradient method (step S3). The model parameters are updated using the gradients stored in the gradient storage unit 60. When step S3 is executed for the first time, the model parameters are not updated.

[0036] Next, the loss function calculation unit 50 adjusts the degree of discrete-continuous (step S4). Specifically, the loss function calculation unit 50 adjusts the hyperparameter λ in the above formula (6) by changing the hyperparameter λ to a negative value λ （０） < 0 to positive value λ （Ｔ） The loss function is calculated while gradually changing it to >0.

[0037] Next, the loss function calculation unit 50 determines whether the convergence condition is satisfied (step S5). For example, it determines whether the loss function of the above formula (6) is no longer smaller than the specified value even when step S4 is repeatedly executed. If the determination in step S5 is "No," the process is executed again from step S3.

[0038] If the determination in step S5 is "Yes," the execution of the flowchart ends. In this case, the model parameter storage unit 20 stores the model parameters that result in the smallest loss function.

[0039] 6, a machine learning model that minimizes the loss function of the above formula (6) can be obtained. The machine learning model (model parameters) is stored in the model parameter storage unit 20.

[0040] Fig. 7 is a flowchart showing an example of the operation of the information processing device 100 when an approximate solution to an optimization problem is output using the results of the machine learning model obtained by the machine learning in Fig. 6. As illustrated in Fig. 7, the node embedding unit 30 embeds the optimization problem (step S11).

[0041] Next, the approximate solution output unit 70 acquires the output of the machine learning model (step S12).

[0042] Next, the approximate solution output unit 70 performs threshold processing on the optimal solution output by the machine learning model (step S13). For example, a threshold is set for each value output by the machine learning model to binarize it. For example, when converting each value into two values ​​of 0 and 1, the threshold is set to 0.5, and values ​​greater than 0.5 are set to 1 and values ​​less than 0.5 are set to 0.

[0043] In the above embodiment, an optimization problem using a graph as the optimization target has been described. Optimization problems using a graph are not particularly limited, but an example thereof is an energy transportation problem. The above embodiment can also be applied to optimization problems that do not use a graph as the optimization target. Optimization problems that do not use a graph are also not particularly limited, but an example thereof is a corporate scheduling problem.

[0044] In the above embodiment, the loss function calculation unit 50 functions as an example of an execution unit that executes a process of machine learning a model by using a loss term according to the degree of continuity or discreteness of the variable to be optimized as a cost function in a search process that performs a search by incorporating continuous relaxation into a discrete optimization problem, and by changing the loss term according to the progress of the search process. The approximate solution output unit 70 functions as an example of an output unit that outputs a solution by embedding the optimization problem in a machine-learned model.

[0045] Although the embodiments of the present invention have been described in detail above, the present invention is not limited to such specific embodiments, and various modifications and changes are possible within the scope of the gist of the present invention as described in the claims.

[0046] REFERENCE SIGNS LIST 10 Optimization problem storage unit 20 Model parameter storage unit 30 Node embedding unit 40 Relaxation variable unit 50 Loss function calculation unit 60 Gradient storage unit 70 Approximate solution output unit 100 Information processing device 101 CPU 102 RAM 103 Storage device 104 Input device 105 Display device

Claims

1. A machine learning program that causes a computer to execute a process of machine learning a model by using a loss term according to the degree of continuity and discreteness of the variable to be optimized as a cost function in a search process that incorporates continuous relaxation into a discrete optimization problem, and by changing the loss term as the search process progresses.

2. The machine learning program of claim 1, characterized in that the computer is caused to execute a process of changing the loss term as the search process progresses from one in which the loss is smaller the more continuous the variable is to one in which the loss is larger the more continuous the variable is.

3. The machine learning program of claim 1, wherein the discrete optimization problem is expressed in QUBO format.

4. A judgment program that causes a computer to execute a process of outputting a solution by embedding an optimization problem in the model obtained by executing the machine learning program described in any one of claims 1 to 3.

5. A machine learning method, characterized in that a computer executes a process of machine learning a model by using a loss term according to the degree of continuity and discreteness of the variable to be optimized as a cost function in a search process that incorporates continuous relaxation into a discrete optimization problem, and by changing the loss term as the search process progresses.

6. The machine learning method described in claim 5, characterized in that the computer executes a process of changing the loss term as the search process progresses from one that results in a smaller loss the more continuous the variable is to one that results in a larger loss the more continuous the variable is.

7. The machine learning method of claim 5, wherein the discrete optimization problem is expressed in QUBO format.

8. A determination method comprising: a computer executing a process for outputting a solution by embedding an optimization problem in the model obtained by executing the machine learning method described in any one of claims 5 to 7.

9. A machine learning device comprising an execution unit that executes a process of machine learning a model by using a loss term according to the degree of continuity and discreteness of the variable to be optimized as a cost function in a search process that incorporates continuous relaxation into a discrete optimization problem, and by changing the loss term according to the progress of the search process.

10. The machine learning device described in claim 9, characterized in that the execution unit changes the loss term as the search process progresses from one in which the loss is smaller the more continuous the variable is to one in which the loss is larger the more continuous the variable is.

11. The machine learning device according to claim 9, wherein the discrete optimization problem is expressed in QUBO format.

12. A determination device comprising an output unit that outputs a solution by embedding an optimization problem in the model machine-learned by the machine learning device according to any one of claims 9 to 11.

Images