Parameter identification program, parameter identification method, and information processing apparatus

Abstract

To efficiently identify values of a plurality of parameters used in an optimization algorithm.SOLUTION: A parameter identification program causes a computer to execute the processes of: identifying one or more first parameters of a plurality of parameters used in an optimization algorithm, the one or more first parameters having a top degree of contribution to an optimization problem; identifying respective one or more first values of the one or more first parameters while respective values of one or more second parameters in the plurality of parameters are set to a given value, the one or more second parameters being different from the one or more first parameters; and identifying respective one or more second values of the one or more second parameters while values of the one or more first parameters are set to the first values.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to a parameter specifying program, a parameter specifying method, and an information processing device. [Background technology]

[0002] Optimization algorithms such as simulated annealing (SA) and tabu search (TS) are known as methods for solving optimization problems such as combinatorial optimization problems.

[0003] In such optimization algorithms, tuning is sometimes performed to appropriately set parameters, such as hyperparameters, used when searching for an optimal solution. In this tuning, the appropriate setting of parameters is performed for each problem (instance) for which an optimal solution is to be searched. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 8-272761 [Patent Document 2] Japanese Patent Application Laid-Open No. 2001-124667 [Patent Document 3] US Patent Application Publication No. 2017 / 0161612 [Patent Document 4] US Patent Application Publication No. 2006 / 0036561 Summary of the Invention [Problem to be solved by the invention]

[0005] Tuning requires multiple trial searches to find appropriate parameter settings, which takes time. Furthermore, when there are multiple parameters to search for an optimal solution, it is difficult to simultaneously identify all of these parameters. For example, when multiple parameters are interrelated, in other words, when multiple parameters cannot be set independently of each other, it becomes even more difficult to simultaneously identify all of these parameters.

[0006] In one aspect, an object of the present invention is to efficiently specify values ​​of multiple parameters used in an optimization algorithm. [Means for solving the problem]

[0007] In one aspect, the parameter identification program may cause a computer to execute the following processes. The process may identify one or more first parameters from among a plurality of parameters used in an optimization algorithm, the first parameters having the highest contribution to the optimization problem. The process may also identify a first value for each of one or more second parameters, which are different from the one or more first parameters among the plurality of parameters, while setting each of the second parameters to a predetermined value. The process may also identify a second value for each of the one or more second parameters while setting each of the first parameters to the first value. [Effects of the Invention]

[0008] In one aspect, values ​​for multiple parameters used in an optimization algorithm can be efficiently identified. [Brief explanation of the drawings]

[0009] [Figure 1] 10 is a flowchart illustrating a parameter tuning technique according to a comparative example. [Figure 2] FIG. 10 is a diagram illustrating an example of the relationship between parameter values ​​and evaluation function values ​​according to contribution degrees. [Figure 3] FIG. 2 is a block diagram illustrating an example of a hardware configuration of a computer that realizes the functions of an optimization device according to an embodiment. [Figure 4] FIG. 2 is a block diagram illustrating an example of the software configuration of an optimization device according to an embodiment. [Figure 5] FIG. 10 is a diagram illustrating an example of a selection method for a parameter set A according to an embodiment. [Figure 6] 10 is a flowchart illustrating an example of the operation of an optimization device according to an embodiment. [Figure 7] FIG. 10 is a diagram illustrating an example of calculation of the contribution degree by the first method. [Figure 8] FIG. 10 is a diagram showing an example of an evaluation function value according to the first method. [Figure 9] FIG. 10 is a diagram illustrating an example of ranges α and β. [Figure 10] FIG. 10 is a diagram for explaining an example of calculation of ranges α and β. [Figure 11] FIG. 10 is a diagram illustrating an example of tuning time according to the second technique. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the embodiments described below are merely examples, and are not intended to exclude various modifications or application of techniques not explicitly described below. For example, the present embodiment can be implemented with various modifications within the scope of the spirit thereof. In the drawings used in the following description, parts with the same reference numerals represent the same or similar parts unless otherwise specified.

[0011] [A] Parameter tuning First, tuning of multiple parameters used in the optimization algorithm will be explained.

[0012] FIG. 1 is a flowchart illustrating a parameter tuning method according to a comparative example. FIG. 1 illustrates an example of a tuning method executed by an optimization device that solves an optimization problem, in which an optimal parameter combination is found by repeatedly selecting combinations of all parameters to be tuned. The parameters are, for example, hyperparameters. Hereinafter, this method may be referred to as a "batch parameter tuning method."

[0013] In step S101, an instance is input to the optimization device. An instance is a specific example or case of a particular problem in the optimization problem. The batch parameter tuning method is executed for each instance for which an optimal solution is to be searched. The optimization device executes parameter tuning processing S110 (S102 to S104) for the input instance.

[0014] In step S102, the optimization device selects values ​​for each of the multiple parameters from candidate values ​​that can be set for each parameter to be tuned (candidate values ​​that each parameter can take), i.e., selects a combination of parameter values.

[0015] In step S103, the optimization device sets the values ​​of a plurality of parameters in the selected combination, executes a process for solving the optimization problem for the instance, and evaluates the parameters in the selected combination based on the execution result.

[0016] In step S104, the optimization device determines whether or not to end the parameter tuning. If not to end (NO in step S104), the process proceeds to step S102. If to end (YES in step S104), the process proceeds to step S105. Whether or not to end the parameter tuning may be determined based on, for example, whether or not an evaluation result (e.g., an evaluation function value) that satisfies an end criterion has been obtained in step S103, or whether or not a predetermined number of repetitions have been performed. The evaluation function value may mean the value of the function (evaluation function) to be optimized.

[0017] In step S105, the optimization device outputs the optimum combination of parameter values, for example, the combination of parameter values ​​that has produced the best result in the parameter evaluation, and the process ends.

[0018] In tuning techniques, it is important to find more appropriate parameter combinations and improve the quality of solutions in order to improve the solution-finding performance of the optimization device.

[0019] Here, in the collective parameter tuning method according to the comparative example shown in FIG. 1, it is assumed that the parameters are tuned under the assumption that the parameters are independent of each other.

[0020] Under such assumptions, if there is actually a correlation between the parameters, using the batch parameter tuning method may result in the following situations regarding the parameter contribution and the evaluation function value: The evaluation function value may mean the value of the function (evaluation function) to be optimized. The parameter contribution may mean the magnitude of change in the evaluation function value in response to a change in the parameter value.

[0021] 2 is a diagram showing an example of the relationship between parameter values ​​and evaluation function values ​​according to the degree of contribution. For example, when there is actually a correlation between the parameters, a parameter with a high degree of contribution such as that indicated by symbol A1 may be correlated with a parameter with a low degree of contribution such as that indicated by symbol A2, resulting in a deterioration in the evaluation function value and other instability in the evaluation function value.

[0022] Furthermore, for example, a parameter that has a stable and high contribution to the instance for which the optimal solution is being searched may be correlated with a parameter whose contribution varies from instance to instance, resulting in a deterioration in the evaluation function value, and the evaluation function value may become unstable.

[0023] In this way, when multiple parameters are interrelated when searching for an optimal solution, in other words, when multiple parameters cannot be set independently of each other, it may be difficult to simultaneously identify all of these multiple parameters.

[0024] Therefore, in one embodiment, a method for efficiently specifying values ​​of multiple parameters used in an optimization algorithm is described.

[0025] [B] Configuration example of an optimization device according to one embodiment An example of the configuration of an optimization device 1 (see FIG. 4) according to an embodiment will be described below.

[0026] [B-1] Hardware configuration example The optimization device 1 according to one embodiment may be a virtual server (VM: Virtual Machine) or a physical server. The functions of the optimization device 1 may be implemented by one computer or two or more computers. Furthermore, at least some of the functions of the optimization device 1 may be implemented using hardware (HW) resources and network (NW) resources provided by a cloud environment.

[0027] 3 is a block diagram showing an example of the hardware (HW) configuration of a computer 10 that realizes the functions of the optimization device 1 as an example of an embodiment. When multiple computers are used as HW resources that realize the functions of the optimization device 1, each computer may have the HW configuration shown in FIG.

[0028] As shown in FIG. 3, the computer 10 may, as a hardware configuration, illustratively include a processor 10a, a graphics processing unit 10b, a memory 10c, a storage unit 10d, an IF (Interface) unit 10e, an IO (Input / Output) unit 10f, and a reading unit 10g.

[0029] The processor 10a is an example of a processing unit that performs various controls and calculations. The processor 10a may be connected to each block in the computer 10 via a bus 10j so that they can communicate with each other. The processor 10a may be a multiprocessor including multiple processors, a multi-core processor having multiple processor cores, or a configuration having multiple multi-core processors.

[0030] The processor 10a may be, for example, an integrated circuit (IC) such as a CPU, MPU, APU, DSP, ASIC, or FPGA. Note that the processor 10a may be a combination of two or more of these integrated circuits. CPU is an abbreviation for Central Processing Unit, MPU is an abbreviation for Micro Processing Unit, APU is an abbreviation for Accelerated Processing Unit, DSP is an abbreviation for Digital Signal Processor, ASIC is an abbreviation for Application Specific IC, and FPGA is an abbreviation for Field-Programmable Gate Array.

[0031] The graphics processing device 10b controls screen display for an output device such as a monitor in the IO unit 10f. The graphics processing device 10b may also be configured as an accelerator that executes machine learning processing and inference processing using a machine learning model. The graphics processing device 10b may be various types of arithmetic processing devices, such as a GPU (Graphics Processing Unit), an APU, a DSP, an ASIC, an FPGA, or other integrated circuits (ICs).

[0032] The memory 10c and the storage unit 10d each store various types of data, programs, and other information. Examples of the memory 10c include one or both of a volatile memory such as a dynamic random access memory (DRAM) and a nonvolatile memory such as a persistent memory (PM). Examples of the storage unit 10d include various storage devices such as a magnetic disk device such as a hard disk drive (HDD), a semiconductor drive device such as a solid state drive (SSD), and a nonvolatile memory. Examples of the nonvolatile memory include a flash memory, a storage class memory (SCM), and a read-only memory (ROM).

[0033] The storage unit 10d may store a program 10h (parameter identification program) that realizes all or part of the various functions of the computer 10. For example, the processor 10a of the optimization device 1 can realize the function of a control unit 7 (see FIG. 4), which will be described later, by expanding the program 10h stored in the storage unit 10d into the memory 10c and executing it.

[0034] The IF unit 10e is an example of a communication IF that controls the connection and communication between the optimization device 1 and other computers. For example, the IF unit 10e may include an adapter that complies with electrical communications (e.g., LAN (Local Area Network)) such as Ethernet (registered trademark) or optical communications such as FC (Fibre Channel). The adapter may support one or both of wireless and wired communication methods. The program 10h may be downloaded to the computer 10 from a network via the communication IF and stored in the storage unit 10d.

[0035] The IO unit 10f may include one or both of an input device and an output device. Examples of input devices include a keyboard, a mouse, and a touch panel. Examples of output devices include a monitor, a projector, and a printer. The IO unit 10f may also include a touch panel that combines an input device and an output device. The output device may be connected to the graphics processing device 10b.

[0036] The reading unit 10g is an example of a reader that reads data and program information recorded on the recording medium 10i. The reading unit 10g may include a connection terminal or device to which the recording medium 10i can be connected or inserted. Examples of the reading unit 10g include an adapter that complies with USB (Universal Serial Bus) or the like, a drive device that accesses a recording disk, and a card reader that accesses a flash memory such as an SD card. Note that the recording medium 10i may store the program 10h, and the reading unit 10g may read the program 10h from the recording medium 10i and store it in the memory unit 10d.

[0037] Examples of the recording medium 10i include non-transitory computer-readable recording media such as magnetic / optical disks and flash memories. Examples of magnetic / optical disks include flexible disks, CDs (Compact Discs), DVDs (Digital Versatile Discs), Blu-ray Discs, and HVDs (Holographic Versatile Discs). Examples of flash memories include semiconductor memories such as USB memories and SD cards.

[0038] The above-described HW configuration of the computer 10 is an example. Therefore, the HW in the computer 10 may be increased or decreased (for example, adding or deleting any block), divided, integrated in any combination, or the HW may be added or deleted as needed.

[0039] [B-2] Software configuration example 4 is a block diagram showing an example of the software configuration of an optimization device 1 according to an embodiment. The optimization device 1 is an example of a computer or information processing device, and is an example of a parameter identification device that executes a parameter identification process to identify values ​​of multiple parameters used in an optimization algorithm. The optimization device 1 may also execute a process for solving an optimization problem, such as a combinatorial optimization problem, for an instance.

[0040] In the following description, the optimization device 1 executes a solution process for an optimization problem for an instance using parameters identified by the parameter identification process, for example, hyperparameters, but is not limited to this. The optimization device 1 may be, for example, a device that executes the parameter identification process out of the parameter identification process and the solution process, and may output (provide) the identified parameters to another optimization device that executes the solution process.

[0041] As illustrated in Fig. 4, the optimization device 1 includes a memory unit 2, a contribution calculation unit 3, an order determination unit 4, a tuning processing unit 5, and an output unit 6. The contribution calculation unit 3, the order determination unit 4, the tuning processing unit 5, and the output unit 6 are examples of a control unit 7. The control unit 7 may execute a process for solving an optimization problem. The functions of the control unit 7 may be realized, for example, by a processor 10a of a computer 10 shown in Fig. 3 executing a program 10h loaded in a memory 10c.

[0042] The memory unit 2 stores various data used in the execution of the parameter identification process by the optimization device 1. For example, the memory unit 2 may be capable of storing performance information 2a, instance information 2b, and optimal parameter values ​​2c. The memory unit 2 may be realized, for example, by the storage areas of one or both of the memory 10c and the storage unit 10d of the computer 10 shown in FIG. 3.

[0043] The performance information 2a is past results (experimental results), for example, parameter tuning results previously performed on multiple instances. The performance information 2a includes, for each instance, a set (pair) of each value (combination of parameter values) of multiple parameters (hyperparameters) identified for each of the multiple instances, and an evaluation function value obtained when the combination is set. Note that the multiple instances all belong to a common field, but the problems (optimization problems) and the solutions obtained by solving the problems may be different from one another. An example of a field is IT (Information Technology) drug discovery.

[0044] In the performance information 2a, the parameter tuning processes executed in the past may be the method according to one embodiment, or another method (for example, a method according to a comparative example), or these may be mixed. Furthermore, the performance information 2a may be calculated by the optimization device 1, or may be calculated by another optimization device.

[0045] The instance information 2b is information about an instance that is the search target for an optimal solution, for example, a specific example or case of a particular problem in the optimization problem, in other words, information indicating the conditions for the solution process of the optimization problem.

[0046] The optimization device 1 (control unit 7) may receive, for example, at least one of the performance information 2a and the instance information 2b from another computer (not shown) via the IF unit 10e and the network, and store it in a storage area.

[0047] The contribution calculation unit 3 calculates the contribution of each parameter to a change in the evaluation function based on the performance information 2a. The contribution is an example of an index that indicates the degree to which a change in the parameter value contributes to a change in the evaluation function value.

[0048] The order determination unit 4 determines the order of tuning of the plurality of parameters based on the contribution calculated by the contribution calculation unit 3.

[0049] For example, the order determination unit 4 selects parameter group A, which has a stable and high degree of contribution to changes in the evaluation function across multiple instances. Parameter group A is an example of a first parameter, which has a degree of contribution to the optimization problem that is ranked at least one higher among multiple parameters used in the optimization algorithm.

[0050] Fig. 5 is a diagram illustrating an example of a selection method for parameter group A according to an embodiment. Fig. 5 shows the contribution of each of parameters #0 to #3 to each of instances #0 to #2. In the example shown in Fig. 5, the contribution of parameter #0 is the most stable and highest in each of instance #0, instance #1, and instance #2.

[0051] In this case, the order determination unit 4 selects parameter #0 as parameter group A. Note that the order determination unit 4 may treat the remaining parameters #1, #2, and #3 of the multiple parameters other than parameter group A as parameter group B. Parameter group B is an example of one or more second parameters that are different from one or more first parameters of the multiple parameters. Parameter groups A and B may each include one or more parameters.

[0052] The order determination unit 4 determines the parameter group A as the parameters to be tuned first (first), and determines the parameter group B as the parameters to be tuned next (last), thereby determining the order in which the multiple parameters are tuned (set to appropriate values).

[0053] The tuning processing unit 5 executes step-by-step tuning of a plurality of parameters in accordance with the order determined by the order determination unit 4.

[0054] The tuning processing unit 5 specifies each value of the parameter group A by performing tuning of the parameter group A (tuning process A) while setting (e.g., fixing) each value of the parameter group B to a predetermined value. The predetermined value may be, for example, a preset value (default value). Each value of the parameter group A is an example of a first value.

[0055] For example, the tuning processing unit 5 selects, as a combination of parameter values, each value of parameter group B set to a predetermined value and each value of parameter group A selected from candidate values ​​that can be set for each of parameter group A (candidate values ​​that each parameter can take). The tuning processing unit 5 sets the values ​​of multiple parameters in the selected combination, executes a process for solving the optimization problem for the instance, and evaluates the parameters in the selected combination based on the execution results. The tuning processing unit 5 completes tuning processing A by executing the process for solving and evaluating the parameters for all combinations obtained by changing the values ​​of parameter group A.

[0056] Furthermore, the tuning processing unit 5 specifies each value of parameter set B by tuning parameter set B while each value of parameter set A is set to a specified value (for example, fixed). Each value of parameter set B is an example of a second value.

[0057] For example, the tuning processing unit 5 selects, as a combination of parameter values, each value of the identified parameter group A and each value of parameter group B selected from candidate values ​​that can be set for each of parameter group B (candidate values ​​that each parameter can take). The tuning processing unit 5 sets the values ​​of multiple parameters in the selected combination, executes a process for solving the optimization problem for the instance, and evaluates the parameters in the selected combination based on the execution results. The tuning processing unit 5 completes tuning processing B by executing a process for solving the problem and evaluating the parameters for all combinations obtained by changing the values ​​of parameter group B.

[0058] The tuning process by the tuning processor 5 may be the same as the parameter evaluation process (step S103) shown in FIG. 1, for example.

[0059] In this way, the tuning processing unit 5 first tunes the identified parameter group A among the multiple parameters to be tuned, and then tunes parameter group B.

[0060] The output unit 6 outputs (stores) each value of parameter set A identified by the tuning processing unit 5 and each value of parameter set B identified by the tuning processing unit 5 as optimal parameter values ​​2c in the memory unit 2. The optimal parameter values ​​2c are a combination of optimal parameter values ​​and are used, for example, as setting values ​​of hyperparameters when a solution process for an optimization problem is executed. The output unit 6 may transmit the optimal parameter values ​​2c to another computer (not shown) via the IF unit 10e and a network.

[0061] [C] Operational example of one embodiment 6 is a flowchart illustrating an example of the operation of the optimization device 1 according to an embodiment. It is assumed that the memory unit 2 stores performance information 2a and instance information 2b.

[0062] In step S1, the contribution calculation unit 3 calculates the contribution of each parameter based on the performance information 2a.

[0063] In step S2, the order determination unit 4 determines the tuning order, for example, parameter groups A and B, based on the contribution of each parameter.

[0064] In step S3, the instance is input to the tuning processor 5 from the instance information 2b.

[0065] In step S4 (S41 to S43), the tuning processing unit 5 executes tuning processing A on parameter group A.

[0066] In step S41, the tuning processing unit 5 selects one combination of each value of the parameter group A and each value of the parameter group B that is fixed to a predetermined value.

[0067] In step S42, the tuning processing unit 5 evaluates the parameters in the selected combination.

[0068] In step S43, tuning processing unit 5 determines whether or not to end tuning of parameter group A. If not to end (NO in step S43), the process proceeds to step S41. If to end (YES in step S43), the process proceeds to step S5. Whether or not to end parameter tuning may be determined based on, for example, whether or not an evaluation result (e.g., an evaluation function value) that satisfies an end criterion has been obtained in step S42, or whether or not a predetermined number of repetitions have been performed.

[0069] After step S4 is completed, the tuning processing unit 5 executes tuning processing B on parameter group B in step S5 (S51 to S53).

[0070] In step S51, the tuning processing unit 5 selects one combination of each value of the parameter group B and each value of the parameter group A that has been fixed to the value specified in the tuning processing A.

[0071] In step S52, the tuning processing unit 5 evaluates the parameters in the selected combination.

[0072] In step S53, tuning processing unit 5 determines whether or not to end tuning of parameter group B. If not to end (NO in step S53), the process proceeds to step S51. If to end (YES in step S53), the process proceeds to step S6. Whether or not to end parameter tuning may be determined based on, for example, whether or not an evaluation result (e.g., an evaluation function value) that satisfies an end criterion has been obtained in step S52, or whether or not a predetermined number of repetitions have been performed.

[0073] In step S6, the output unit 6 outputs the optimal parameter value combination identified by the tuning processing unit 5, for example, the parameter value combination that provides the best result in the parameter evaluation, as the optimal parameter value 2c, and the process ends. The optimal parameter value 2c may be stored in the memory unit 2, for example.

[0074] As described above, the optimization device 1 according to one embodiment can identify appropriate values ​​for each of a plurality of parameters, even when two or more of the parameters are interrelated. Furthermore, tuning parameter group A before parameter group B reduces the variability in the contribution of parameter group B between instances, allowing the evaluation function value to be a stable, good value, thereby improving the quality of the solution. In this way, the step-by-step tuning method can efficiently identify the values ​​of the multiple parameters used in the optimization algorithm.

[0075] [D] Method for calculating contribution and determining order Next, two examples of a first method and a second method will be described as methods for carrying out the contribution calculation process by the contribution calculation section 3 and the order determination process by the order determination section 4.

[0076] [D-1] First method The first method is a method in which, in the contribution calculation process, an evaluation algorithm for evaluating the contribution of a parameter is used to calculate the contribution of each parameter. Hereinafter, fANOVA (functional analysis of variance) will be described as an example of such an evaluation algorithm. fANOVA is a type of analysis of variance, and is an algorithm for evaluating the contribution of hyperparameters.

[0077] The contribution degree calculation unit 3 may calculate the contribution degree of each parameter by executing processing based on the fANOVA algorithm using the performance information 2a as input data. For example, the contribution degree calculation unit 3 may evaluate the contribution degree of the parameter using fANOVA by the following processing.

[0078] The contribution calculation unit 3 uses the performance information 2a to train a machine learning model (hereinafter, sometimes simply referred to as a "model") that learns the relationship between the values ​​of multiple parameters and the evaluation function values ​​obtained when these parameters are set. For example, a random forest may be used to train the model.

[0079] Based on the trained model, the contribution calculation unit 3 evaluates the contribution (importance) of each parameter to the evaluation function value. In this evaluation, the contribution calculation unit 3 quantitatively evaluates the degree to which each parameter contributes to the evaluation function by using fANOVA.

[0080] In the order determination process according to the first method, the order determination unit 4 extracts one or more parameters with the highest contribution for each instance. Then, the order determination unit 4 identifies, among the extracted parameters, parameters that have been extracted many times in multiple instances as parameters with consistently high contributions in multiple instances, in other words, as parameter group A.

[0081] For example, the order determination unit 4 extracts the top K parameters with high contributions for each instance. The order determination unit 4 selects the top N parameters in descending order of the number of times they have been extracted as high-contribution parameters as a parameter group A to be tuned first. K and N are thresholds, and are both integers equal to or greater than 1. The thresholds K and N may be determined based on the tendency of the distribution of contributions for each instance. For example, if there is a parameter with an exceptionally high contribution, K may be set to 1, N may be set to 1, etc.

[0082] Figure 7 shows an example of calculating the contribution using the first method. Figure 7 shows an example of tuning 10 types of hyperparameters for an optimization problem in IT drug discovery. Note that TPE (Tree-structured Parzen Estimator) was used as the algorithm for selecting the parameter combination. In the results of past experiments on 27 instances (performance information 2a), the parameter with the highest contribution was "parameter #0" in 23 instances. Therefore, "parameter #0" was selected as parameter group A, which has a stable and high contribution.

[0083] FIG. 8 is a diagram showing an example of an evaluation function value obtained by the first method. FIG. 8 shows the results of a comparison between a batch parameter tuning method according to a comparative example (denoted as "batch" in FIG. 8) and a stepwise parameter tuning method according to the first method (denoted as "stepwise" in FIG. 8). In FIG. 8, tuning of parameter group A and tuning of parameter group B in the stepwise parameter tuning method (first method) were performed in approximately half the tuning time of the batch parameter tuning method. In other words, the tuning time of the comparative example was set to be equivalent to the total tuning time of the first method. In FIG. 8, for each of the comparative example and the first method, calculation of the evaluation function value obtained when the optimal combination of parameter values ​​finally identified by tuning was set was performed 10 times, and the median and average of the measurement results were calculated.

[0084] As shown in Figure 8, in each of the examples of instance #0 (symbol B1) and instance #1 (symbol B2), the first method produced better results than the comparative example in both the median and average evaluation function values, confirming its effectiveness in improving the quality of the solution.

[0085] Although FIG. 8 shows the tuning time for parameter set A and the tuning time for parameter set B in the first method as being the same, better effects can be achieved by adjusting the ratio of the tuning time for parameter set A to the tuning time for parameter set B. For example, the tuning processor 5 may reduce the ratio of the tuning time for parameter set A to the overall tuning time depending on the number of hyperparameters to be identified. As an example, the tuning processor 5 may reduce the ratio of the tuning time for parameter set A to the tuning time for parameter set B as the number of hyperparameters to be identified decreases. If optimal tuning times for parameter sets A and B can be determined in this way, better effects can be expected in terms of both quality and time.

[0086] [D-2] Second method The second method is a method in which, in the contribution calculation process, a possible range (first range) α of each parameter and a range (second range) β in which a good evaluation function value is obtained are defined for each instance based on the performance information 2a, and the ratio of range β to range α is calculated as the contribution. A good evaluation function value may be, for example, an evaluation function value equal to or greater than a predetermined threshold. For example, the contribution calculation unit 3 may identify range β by determining parameter values ​​that yield good solutions equal to or greater than a predetermined threshold.

[0087] 9 is a diagram showing an example of range α and range β. In the second technique, the contribution calculation unit 3 may calculate the value of β / α for each parameter in a graph (histogram) in which the horizontal axis represents the parameter value and the vertical axis represents the frequency of the evaluation function value. The contribution calculation unit 3 may determine the contribution of each parameter so that the smaller the value of β / α, the higher the contribution.

[0088] For example, based on the performance information 2a, the contribution calculation unit 3 normalizes the frequency distribution of each parameter for each past instance so that the minimum value is "0" and the maximum value is "1." Then, the contribution calculation unit 3 fits the frequency distribution of parameter values ​​for which good evaluation function values ​​were obtained in multiple past instances to a normal distribution.

[0089] In this way, the second method selects parameter group A based on the knowledge that the steeper the peak shape in the frequency distribution for a parameter value, the narrower the range that the parameter can actually take, and that such parameters tend to have a greater contribution.

[0090] Below, we will explain an example of calculating the ranges α and β by the contribution degree calculation unit 3 according to the status of the parameters. Note that the calculation (identification) of the ranges α and β and the calculation of the contribution degree based on the ranges α and β are performed for each parameter to be identified.

[0091] (Example 1) When the minimum and maximum values ​​of the parameters are not known in advance. In this case, the contribution calculation unit 3 fits the frequency distribution of the parameter values ​​obtained from the tuning results of the past instance to a normal distribution N1, and sets the mean value of the normal distribution N1 to μ1 and the standard deviation to σ1. Then, the contribution calculation unit 3 normalizes the range α to the range of "μ1±6σ1" so that the minimum value is "0" and the maximum value is "1". Then, the contribution calculation unit 3 fits the frequency distribution of the parameter values ​​for which good evaluation function values ​​were obtained in the past instance to a normal distribution N2, and sets the mean value of the normal distribution N2 to μ2 and the standard deviation to σ2. Then, the contribution calculation unit 3 sets the range of "μ2±σ2" to the range β.

[0092] (Example 2) The minimum and maximum values ​​of a parameter are known in advance, and the frequency distribution of parameter values ​​obtained from tuning results of past instances within the range from the minimum to the maximum value is not biased towards the minimum or maximum value. In this case, as illustrated in Fig. 9, the contribution calculation unit 3 normalizes the parameter so that the minimum value is "0" and the maximum value is "1" within the range α, from the minimum value to the maximum value of the parameter. Then, the contribution calculation unit 3 fits the frequency distribution of the parameter values ​​for which good evaluation function values ​​were obtained in past instances with a normal distribution N2, and sets the mean value of the normal distribution N2 to μ2 and the standard deviation to σ2. Then, the contribution calculation unit 3 sets the range of "μ2 ± σ2" to the range β.

[0093] (Example 3) The minimum and maximum values ​​of a parameter are known in advance, and the frequency distribution of parameter values ​​obtained from tuning results of past instances is biased toward the minimum or maximum value within the range from the minimum to maximum value. FIG. 10 is a diagram illustrating an example of calculating the range α and the range β. Reference symbol C1 in FIG. 10 illustrates a case where the frequency distribution of parameter values ​​is biased toward the minimum value side. As indicated by reference symbol C2, the contribution calculation unit 3 widens the range toward the biased side of the distribution, fits the frequency distribution to a normal distribution N1, and sets μ1 as the mean value and σ1 as the standard deviation of the normal distribution N1. Then, the contribution calculation unit 3 normalizes the range α, which is the range of μ1±3σ1, so that the minimum value is “0” and the maximum value is “1.” In this way, the range α, which is the range of μ1±3σ1, is obtained from the frequency distribution indicated by reference symbol C1, as indicated by reference symbol C2. Then, as indicated by reference symbol C3, the contribution calculation unit 3 fits the frequency distribution of parameter values ​​for which good evaluation function values ​​were obtained in past instances to a normal distribution N2, and sets μ2 as the mean value and σ2 as the standard deviation of the normal distribution N2. In the histogram C3, the shaded area represents the frequency distribution of the evaluation function values ​​that are better than a predetermined threshold, and the solid line represents the normal distribution N2 obtained from the shaded area. From this normal distribution, the range β (μ2±σ2) is identified.

[0094] The contribution calculation unit 3 defines the contribution as the ratio β / α of the size of range β to the size of range α identified by the above method, and defines the contribution as higher as the ratio β / α. In the above example, the size of range α is normalized to 1, and the size of range β is 2σ2, so β / α = β = 2σ2. In other words, the contribution calculation unit 3 may define the contribution as β, which is twice the standard deviation σ2 of the normal distribution N2, and defines the contribution as higher as the standard deviation σ2 is smaller.

[0095] In the order determination process according to the second method, the order determination unit 4 extracts one or more parameters with the highest contribution, for example, the top K parameters, for each instance. The order determination unit 4 calculates the variance of the mean value μ2 over multiple instances based on the mean value μ2 of the normal distribution N2 fitted by the contribution calculation unit 3 for each extracted parameter.

[0096] Then, the order determination unit 4 identifies, from among the extracted parameters with high contribution, parameters with small variance of the average value μ2 as parameters with stably high contribution across multiple instances, in other words, as parameter group A. For example, from among the extracted parameters with high contribution, the order determination unit 4 identifies one or more parameters, for example the top N parameters, in order of smallest variance of the average value μ2 as parameter group A. The thresholds K and N are the same as those described in the first method.

[0097] As described above, the second method can also achieve the same effect as the first method (for example, to the same extent).

[0098] Fig. 11 is a diagram showing an example of tuning time using the second method. Fig. 11 shows a comparison of processing times between a batch parameter tuning method according to a comparative example (denoted as "batch" in Fig. 11) and a stepwise parameter tuning method according to the second method (denoted as "stepwise" in Fig. 11). As shown in Fig. 11, the second method can obtain an evaluation function value equivalent to that of the method according to the comparative example in a tuning time that is up to five times faster (one-fifth or less), confirming the effectiveness of shortening tuning time by improving the quality of the solution.

[0099] [E] Other The technology according to the embodiment described above can be implemented with the following modifications and variations.

[0100] For example, the contribution calculation unit 3, order determination unit 4, tuning processing unit 5, and output unit 6 included in the optimization device 1 shown in FIG. 4 may be combined in any combination, or may be separated.

[0101] In addition, in one embodiment, an example of a stepwise parameter tuning method in which multiple parameters are divided into parameter group A and parameter group B and two-stage tuning is performed has been described, but the present invention is not limited to this. For example, the optimization device 1 may select parameter group B1 from multiple parameters included in parameter group B, which has a stable and high degree of contribution in multiple instances. In this case, the optimization device 1 may perform three-stage tuning in the order of parameter group A, parameter group B1, and parameter group B2. Four or more stages of tuning are also possible by further dividing the remaining parameter group (for example, parameter group B2).

[0102] Furthermore, the known evaluation algorithm used in the first method, which can evaluate the contribution of parameters, is not limited to fANOVA, and various other evaluation algorithms may be used. For example, PFI (Permutation Feature Importance), PDP (Partial Dependence Plot), etc. may be used as the evaluation algorithm.

[0103] PFI is an evaluation algorithm that evaluates the contribution (importance) of features by randomly rearranging the values ​​of the features (hyperparameters) and calculating the degree to which the predictive accuracy of the machine learning model decreases. By using PFI, it is possible to understand (judge) the degree to which each feature contributes to the evaluation function.

[0104] PDP is an evaluation algorithm that evaluates the impact of certain features (hyperparameters) by visually displaying how the predictions of a machine learning model change when the value of the feature is changed. By using PDP, it is possible to understand (judge) how each feature affects the evaluation function.

[0105] 4 may be configured such that each processing function is realized by a plurality of devices cooperating with each other via a network. As an example, the contribution calculation unit 3, the order determination unit 4, the tuning processing unit 5, and the output unit 6 may be realized by an application server or a web server, and the memory unit 2 may be realized by a DB (Database) server. In this case, the web server, the application server, and the DB server may cooperate with each other via a network to realize the processing function of the optimization device 1.

[0106] [F] Notes The following additional notes are further disclosed regarding the above embodiment.

[0107] (Appendix 1) identifying a first parameter having a degree of contribution to the optimization problem that is at least one of the most significant parameters among a plurality of parameters used in the optimization algorithm; specifying a first value for each of the one or more first parameters in a state where each value of one or more second parameters different from the one or more first parameters among the plurality of parameters is set to a predetermined value; determining a second value for each of the one or more second parameters while setting the value of each of the one or more first parameters to the first value; A parameter-specific program that causes a computer to execute a process.

[0108] (Appendix 2) The process of identifying one or more first parameters includes: a process of calculating the contribution of each of the plurality of parameters using an evaluation algorithm for evaluating the contribution of the parameters; 1. A parameter identification program as described in Appendix 1.

[0109] (Appendix 3) The process of identifying one or more first parameters includes: and calculating, as the degree of contribution, a ratio of a second range in which an evaluation function value equal to or greater than a predetermined threshold is obtained to a first range in which the parameter can be taken, for each of the plurality of parameters, based on a combination of the parameter values ​​previously identified for each of the plurality of instances and an evaluation function value obtained when the combination is set. 1. A parameter identification program as described in Appendix 1.

[0110] (Appendix 4) The process of identifying one or more first parameters includes: extracting, from the plurality of parameters, one or more parameters with the highest calculated contribution for each instance; calculating a variance of the average value of the contribution degree for each of the extracted parameters over the plurality of instances; identifying one or more parameters from the extracted parameters in descending order of the calculated variance as the one or more first parameters, Parameter identification program according to appendix 3.

[0111] (Appendix 5) outputting a combination of the first value of each of the one or more identified first parameters and the second value of each of the one or more identified second parameters as setting values ​​of hyperparameters to be set in the process of solving the optimization problem; 5. The parameter specifying program according to any one of claims 1 to 4, which causes the computer to execute a process.

[0112] (Appendix 6) identifying a first parameter having a degree of contribution to the optimization problem that is at least one of the most significant parameters among a plurality of parameters used in the optimization algorithm; specifying a first value for each of the one or more first parameters in a state where each value of one or more second parameters different from the one or more first parameters among the plurality of parameters is set to a predetermined value; determining a second value for each of the one or more second parameters while setting the value of each of the one or more first parameters to the first value; A parameter identification method in which processing is performed by a computer.

[0113] (Appendix 7) The process of identifying one or more first parameters includes: a process of calculating the contribution of each of the plurality of parameters using an evaluation algorithm for evaluating the contribution of the parameters; Parameter determination method according to appendix 6.

[0114] (Appendix 8) The process of identifying one or more first parameters includes: and calculating, for each of the plurality of parameters, a ratio of a second range in which an evaluation function value equal to or greater than a predetermined threshold is obtained to a first range in which the parameter can be taken, as a contribution of the parameter, based on a combination of the values ​​of the plurality of parameters previously identified for each of the plurality of instances and an evaluation function value obtained when the combination is set. Parameter determination method according to appendix 6.

[0115] (Appendix 9) The process of identifying one or more first parameters includes: extracting, from the plurality of parameters, one or more parameters with the highest calculated contribution for each instance; calculating a variance of the average value of the contribution degree for each of the extracted parameters over the plurality of instances; identifying one or more parameters from the extracted parameters in descending order of the calculated variance as the one or more first parameters, 9. The parameter determination method according to claim 8.

[0116] (Appendix 10) outputting a combination of the first value of each of the one or more identified first parameters and the second value of each of the one or more identified second parameters as setting values ​​of hyperparameters to be set in the process of solving the optimization problem; 10. The parameter specifying method according to any one of Supplementary Note 6 to Supplementary Note 9, wherein the processing is executed by the computer.

[0117] (Appendix 11) identifying a first parameter having a degree of contribution to the optimization problem that is at least one of the most significant parameters among a plurality of parameters used in the optimization algorithm; specifying a first value for each of the one or more first parameters in a state where each value of one or more second parameters different from the one or more first parameters among the plurality of parameters is set to a predetermined value; determining a second value for each of the one or more second parameters while setting the value of each of the one or more first parameters to the first value; An information processing device comprising a control unit.

[0118] (Appendix 12) In the process of identifying the one or more first parameters, the control unit: Calculating the contribution of each of the plurality of parameters using an evaluation algorithm for evaluating the contribution of the parameters; 12. The information processing device according to claim 11.

[0119] (Appendix 13) In the process of identifying the one or more first parameters, the control unit: calculating, for each of the plurality of parameters, a ratio of a second range in which an evaluation function value equal to or greater than a predetermined threshold is obtained to a first range in which the parameter can be taken, based on a combination of the values ​​of the plurality of parameters previously identified for each of the plurality of instances and an evaluation function value obtained when the combination is set, as the contribution of the parameter; 12. The information processing device according to claim 11.

[0120] (Appendix 14) In the process of identifying the one or more first parameters, the control unit: extracting, from the plurality of parameters, one or more parameters with the highest calculated contribution for each instance; calculating a variance of the average value of the contribution degree for each of the extracted parameters over the plurality of instances; Among the extracted parameters, one or more parameters are identified as the one or more first parameters, in descending order of the calculated variance. 14. The information processing device according to claim 13.

[0121] (Appendix 15) The control unit outputting a combination of the first value of each of the one or more identified first parameters and the second value of each of the one or more identified second parameters as setting values ​​of hyperparameters to be set in the process of solving the optimization problem; An information processing device according to any one of Supplementary Note 11 to Supplementary Note 14. [Explanation of symbols]

[0122] 1 Optimization device 10. Computers 2 Memory section 2a Performance Information 2b Instance information 2c Optimal parameter values 3 Contribution calculation section 4 Order determining part 5 Tuning processing section 6 Output section 7 Control Unit

Claims

1. identifying one or more first parameters having a highest degree of contribution to the optimization problem from among a plurality of parameters used in the optimization algorithm; specifying a first value for each of the one or more first parameters in a state where each value of one or more second parameters different from the one or more first parameters among the plurality of parameters is set to a predetermined value; specifying a second value for each of the one or more second parameters while setting each value of the one or more first parameters to the first value; A parameter-specific program that causes a computer to execute a process.

2. The process of identifying the one or more first parameters includes: a process of calculating the contribution of each of the plurality of parameters using an evaluation algorithm for evaluating the contribution of the parameters; The parameter specifying program according to claim 1 .

3. The process of identifying the one or more first parameters includes: and calculating, for each of the plurality of parameters, a ratio of a second range in which an evaluation function value equal to or greater than a predetermined threshold is obtained to a first range in which the parameter can be taken, as a contribution of the parameter, based on a combination of the values ​​of the plurality of parameters previously identified for each of the plurality of instances and an evaluation function value obtained when the combination is set. The parameter specifying program according to claim 1 .

4. The process of identifying the one or more first parameters includes: extracting, from the plurality of parameters, one or more parameters with the highest calculated contribution for each instance; calculating a variance of the average value of the contribution degree for each of the extracted parameters over the plurality of instances; and identifying, from among the extracted parameters, one or more parameters in descending order of the calculated variance as the one or more first parameters. The parameter specifying program according to claim 3 .

5. outputting a combination of the first value of each of the one or more identified first parameters and the second value of each of the one or more identified second parameters as setting values ​​of hyperparameters to be set in the process of solving the optimization problem; The parameter specifying program according to any one of claims 1 to 4, which causes the computer to execute processing.

6. identifying one or more first parameters having a highest degree of contribution to the optimization problem from among a plurality of parameters used in the optimization algorithm; specifying a first value for each of the one or more first parameters in a state where each value of one or more second parameters different from the one or more first parameters among the plurality of parameters is set to a predetermined value; specifying a second value for each of the one or more second parameters while setting each value of the one or more first parameters to the first value; A parameter identification method in which processing is performed by a computer.

7. identifying one or more first parameters having a highest degree of contribution to the optimization problem from among a plurality of parameters used in the optimization algorithm; specifying a first value for each of the one or more first parameters in a state where each value of one or more second parameters different from the one or more first parameters among the plurality of parameters is set to a predetermined value; specifying a second value for each of the one or more second parameters while setting each value of the one or more first parameters to the first value; An information processing device comprising a control unit.

Images