Parameter optimization device and parameter optimization method

Abstract

This parameter optimization device (1) is provided with: an evolutionary algorithm unit (2) that uses parameters of an evolutionary algorithm, which are set for each creation of a next-generation subpopulation, to create a next-generation subpopulation that is a next-generation group including fewer individuals than those set for each generation of a target problem, and evaluates the next-generation subpopulation; and an optimization unit (3) that learns creation of the subpopulation on the basis of the result of evaluating the next-generation subpopulation, and selects parameters to be used for creating the next-generation subpopulation on the basis of the result of learning.

Description

Parameter optimization device and parameter optimization method

[0001] The present disclosure relates to a parameter optimization device and a parameter optimization method.

[0002] Evolutionary algorithms are widely used as a technique for solving various optimization problems. For example, Patent Literature 1 (PTL 1) describes a learning method for learning a strategy (π) that optimally adapts at least one parameter (σ) of an evolutionary algorithm, particularly a covariance matrix adaptive evolution strategy (CMA-ES) algorithm or a differential evolutionary algorithm. This method includes the steps of initializing a strategy that calculates a parameter representation (A) of the parameter (σ) depending on state information (S) related to a problem instance, and learning the strategy (π) using reinforcement learning. Then, based on the interaction between the CMA-ES algorithm, the parameter representation determined using the strategy depending on the state information (S), the problem instance, and a reward signal (R), it is learned which parameter representation is optimal for possible state information.

[0003] Japanese Patent Application Laid-Open No. 2022-21177

[0004] In conventional evolutionary algorithm parameter optimization, parameters are selected for each generation, and a set number of individuals (candidate solutions) are generated for each generation using the selected parameters, and then the generated population of individuals is evaluated collectively. Therefore, when the environment fluctuates significantly within a single generation, the evaluation of individuals within a single generation cannot keep up with the environmental changes, resulting in a decrease in the optimality of parameter selection. For example, environmental changes within a single generation can cause changes in the constraints of the problem being solved using the evolutionary algorithm or a change in the objective function. Furthermore, the timing or pattern of evolution can change. These variations in environmental factors have a significant impact on parameter selection.

[0005] In the learning method described in Patent Document 1, the parameter (σ) of the evolutionary algorithm is selected and learned for each generation. Therefore, if the environment changes significantly within one generation, the learning method described in Patent Document 1 is likely to result in a decrease in the optimality of parameter selection and a decrease in the learning efficiency of the evolutionary algorithm.

[0006] The present disclosure is intended to solve the above-mentioned problems, and aims to provide a parameter optimization device that can improve the optimality of parameter selection for an evolutionary algorithm.

[0007] The parameter optimization device according to the present disclosure includes an evolutionary algorithm unit that generates subpopulations, which are a group of next-generation individuals with a number smaller than the number of individuals set for each generation of the target problem, using parameters of an evolutionary algorithm that are set for each generation of the subpopulation, and evaluates the generated subpopulations; and an optimization unit that learns how to generate subpopulations based on the evaluation results of the subpopulations, and selects parameters that the evolutionary algorithm unit will use to generate the subpopulations based on the learning results.

[0008] According to the present disclosure, subpopulations, which are next-generation populations with a number smaller than the number of individuals set for each generation of the target problem, are generated using parameters of an evolutionary algorithm set for each generation of the subpopulations, learning is performed to generate the subpopulations based on evaluation results of the generated subpopulations, and parameters to be used for generating the subpopulations are selected based on the learning results. By selecting parameters and evaluating the subpopulations generated using the parameters for each generation of a subpopulation, the evaluation results of the next-generation subpopulations are fed back into the subpopulation generation process. This improves adaptability to environmental changes for each generation, and the parameter optimization device according to the present disclosure can improve the optimality of parameter selection for the evolutionary algorithm.

[0009] 1A and 1B are schematic diagrams showing an overview of parameter optimization of an evolutionary algorithm. FIG. 1A is a block diagram showing an example configuration of a parameter optimization device according to embodiment 1. FIG. 1B is a block diagram showing an example configuration of a next-generation small population generation unit. FIG. 1C is a flowchart showing a parameter optimization method according to embodiment 1. FIG. 1D is a schematic diagram showing an example of a traveling salesman problem. FIG. 1E is a graph showing a simulation result of optimization of a mutation rate included in a parameter of an evolutionary algorithm for the problem of FIG. 5. FIG. 7A and 7B are block diagrams showing a hardware configuration for realizing the functions of a parameter optimization device according to embodiment 1.

[0010] First Embodiment First, problems with parameter optimization of a conventional evolutionary algorithm will be described with reference to Fig. 1A. Fig. 1A is a schematic diagram showing an overview of parameter optimization of an evolutionary algorithm, illustrating parameter optimization by a conventional parameter optimization device 100. The parameter optimization device 100 includes an evolutionary algorithm unit 101 and an optimization unit 102. The evolutionary algorithm unit 101 generates subpopulations, which are next-generation populations with a set number of individuals for each generation of a target problem, using parameters of the evolutionary algorithm that are set for each generation, and evaluates the generated subpopulations.

[0011] Evolutionary algorithms evolve a population of candidate individuals to obtain an optimal solution by repeating generations of change and selection for the target problem. Evolutionary algorithms are capable of dealing with a variety of optimization problems. However, the performance of evolutionary algorithms depends on parameter tuning, i.e., the ability to select optimal values ​​for hyperparameters such as crossover rate and mutation rate, and operators such as crossover operator and mutation operator.

[0012] In order to improve the optimality of parameter selection, adaptive operator selection (AOS) has been proposed, which solves problems by applying a multi-armed bandit as the evolutionary algorithm unit 101. The parameter optimization device 100 performs the AOS. While learning the parameters of the evolutionary algorithm, the AOS evaluates the individuals generated in the current generation using only historical information about the individual generation executed up to the current generation, and determines the individuals to be retained in the next generation based on the evaluation results. The AOS selects optimal parameters for each generation, making it possible to efficiently control the optimal individual generation.

[0013] For example, in the generation update operation T (processing of generation T), the optimization unit 102 calculates the fitness f Ｔ and obtain the fitness f Ｔ Based on this, the optimal parameters (e.g., crossover rate p Ｔ and the mutation rate e Ｔ The optimization unit 102 sets the selected parameters in the evolutionary algorithm unit 101. For example, the fitness f Ｔ is an index showing how well an individual meets the objective function. Ｔ The higher the individual, the higher the probability that it will be passed on to the next generation.

[0014] The evolutionary algorithm unit 101 performs crossover and point mutation using the parameters of generation T set by the optimization unit 102 to generate a small population of individuals for the next generation T+1. Crossover is an operation that combines the genetic information of two previous-generation individuals (parent individuals) to generate new next-generation individuals (offspring individuals). The evolutionary algorithm unit 101 generates individuals for the next generation T+1 by crossover based on the crossover rate set by the optimization unit 102. Point mutation is an operation that randomly changes part of the genetic information of an individual. The evolutionary algorithm unit 101 generates individuals for the next generation T+1 by point mutation based on the mutation rate set by the optimization unit 102.

[0015] The evolutionary algorithm unit 101 generates a small population of individuals for the next generation T+1, which is made up of individuals with the number of candidate solutions included in the parameters of the generation T, and then performs an evaluation process (fitness evaluation) of this small population to obtain a fitness f Ｔ＋１ to the optimization unit 102. Next, the evolutionary algorithm unit 101 switches from generation T to generation T+1, and in the generation update operation T+1 (processing of generation T+1), generates a small population of individuals for the next generation T+2, consisting of a group of individuals with the number of candidate solutions included in the parameters of generation T+1. The evolutionary algorithm unit 101 generates a small population of individuals for the next generation T+2, and calculates the fitness f Ｔ＋２ to the optimization unit 102. This series of processes is repeated until a preset number of generations is reached.

[0016] As shown in FIG. 1A, in a conventional parameter optimization device 100, parameters are selected for each generation, and information up to the previous generation (the fitness f Ｔ ) is used to generate a next-generation subpopulation consisting of the number of individuals set for that generation, and the generated subpopulations are evaluated collectively. Therefore, if the environment fluctuates significantly in one generation, the evaluation of the individuals in one generation cannot keep up with the environmental fluctuations, and the optimality of parameter selection decreases. In this way, AOS selects and evaluates parameters collectively in each generation, which means that parameter selection is always based only on the evaluation of the previous generation, which is inefficient.

[0017] In contrast, the parameter optimization device according to the first embodiment selects parameters and evaluates the subpopulations generated using the parameters for each subpopulation generation. Fig. 1B is a schematic diagram showing an overview of parameter optimization of an evolutionary algorithm by the parameter optimization device 1 according to the first embodiment. The parameter optimization device 1 includes an evolutionary algorithm unit 2 and an optimization unit 3. The evolutionary algorithm unit 2 generates subpopulations, which are a group of next-generation individuals whose number is smaller than the number of individuals set for each generation of the target problem, using parameters of the evolutionary algorithm that are set for each generation of the subpopulation, and evaluates the generated subpopulations.

[0018] In the generation update process T, the optimization unit 3 calculates the fitness f of a small group of individuals k of the next generation T+1 sequentially generated by the evolutionary algorithm unit 2. Ｔｋ Then, the optimization unit 3 obtains the parameters related to the generation of the small population based on the learning result (for example, the crossover rate p Ｔｋ＋１ and the mutation rate e Ｔｋ＋１ ) and set the selected parameters in the evolutionary algorithm unit 2.

[0019] The evolutionary algorithm unit 2 generates a small population of individuals k of the next generation T+1 using the parameters set by the optimization unit 3, and then performs an evaluation process (fitness evaluation) of this small population to determine the fitness f Ｔｋ The evolutionary algorithm unit 2 then switches from generation T to generation T+1, and in the generation update operation T+1, sequentially generates a small population of individuals for the next generation T+2, which is made up of a group of individuals with a number smaller than the number of candidate solutions included in the parameters of generation T+1. The evolutionary algorithm unit 2 then calculates the fitness f of the generated small population. Ｔ＋１ｋ is output to the optimization unit 3.

[0020] The optimization unit 3 learns how to generate a small population, and based on the learning results, optimizes parameters (for example, crossover rate p Ｔ＋１ｋ＋１ and the mutation rate e Ｔ＋１ｋ＋１ ) is selected and set in the evolutionary algorithm unit 2. This series of processes is repeated until a preset number of individuals for the next generation is reached. In this way, in the parameter optimization device 1, the evaluation results of the next-generation small populations are fed back to the process of generating small populations, improving adaptability to environmental changes in each generation and making it possible to improve the optimality of parameter selection for the evolutionary algorithm.

[0021] Fig. 2 is a block diagram showing an example configuration of the parameter optimization device 1. As shown in Fig. 2, the parameter optimization device 1 is a device equipped with a calculation unit and a storage unit. The calculation unit controls the overall operation of the parameter optimization device 1. The calculation unit includes an evolutionary algorithm unit 2 and an optimization unit 3. The calculation unit executes an information processing application for parameter optimization, thereby realizing various functions of the evolutionary algorithm unit 2 and the optimization unit 3.

[0022] The storage unit stores an information processing application for parameter optimization and information used in the calculation processing of the calculation unit. This storage unit realizes, for example, the evaluation value storage unit 32 and the individual storage unit 213. The storage unit may also be a storage device included in a computer functioning as the parameter optimization device 1. For example, the storage unit may include a storage such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), or the memory 104 shown in FIG. 7B (described later). Note that the storage unit may be provided outside the parameter optimization device 1 as long as it is accessible by the parameter optimization device 1.

[0023] The evolutionary algorithm unit 2 generates next-generation subpopulations, which are a group of next-generation individuals with a number smaller than the number of individuals set for each generation of the target problem, using parameters of the evolutionary algorithm set for each generation of the next-generation subpopulation, and evaluates the next-generation subpopulations. For example, the evolutionary algorithm unit 2 generates individuals for the target problem through N generations from the initial generation, and then generates final-generation individuals with improved fitness. N is an integer equal to or greater than 2.

[0024] The evolutionary algorithm operated in the evolutionary algorithm unit 2 is, for example, a genetic algorithm, a differential evolution algorithm, or a multi-objective optimization algorithm, which are algorithms that perform generation alternation. As shown in FIG. 2 , the evolutionary algorithm unit 2 includes a next-generation generation unit 21 and a generation alternation end determination unit 22.

[0025] The next generation generation unit 21 generates and evaluates the next generation subpopulation for the target problem using an evolutionary algorithm, and includes a parameter setting unit 211, a next generation subpopulation generation unit 212, an individual storage unit 213, an evaluation unit 214, and a generation termination determination unit 215. Furthermore, the generation alternation termination determination unit 22 determines whether or not the next generation generation unit 21 has performed operations from the initial generation to the final generation set in the next generation generation unit 21. For example, if the final generation is the Nth generation, the next generation generation unit 21 generates final generation individuals with improved fitness by repeating N generation alternations from the initial generation until the Nth generation individuals are generated.

[0026] The parameter setting unit 211 sets the parameters acquired from the optimization unit 3 in the next-generation small population generation unit 212. The parameters acquired from the optimization unit 3 are so-called hyperparameters, which are parameters that need to be adjusted when the evolutionary algorithm is operated. For example, the parameters include the number of generations used to identify the final generation in which the next-generation generation unit 21 generates individuals, the number of individuals generated by the next-generation small population generation unit 212, the crossover rate, the mutation rate, and a method for selecting individuals to be used in generating individuals in the next generation. Note that the number of individuals included in the parameters includes not only the number of individuals set for each generation, as in conventional evolutionary algorithms, but also the number of next-generation small populations generated by the next-generation generation unit 21 and the number of individuals in each small population generated by the next-generation small population generation unit 212.

[0027] The next-generation subpopulation generation unit 212 is a generation unit that generates a next-generation subpopulation using the parameters selected by the optimization unit 3. A next-generation subpopulation is a subpopulation made up of multiple individuals that are candidate solutions to the target problem. The next-generation subpopulation includes the number of individuals set for each next-generation subpopulation that are included in the parameters selected by the optimization unit 3. Note that the number of individuals for each next-generation subpopulation is assumed to be less than the number of individuals set for that generation.

[0028] 3 is a block diagram showing an example configuration of the next-generation small population generation unit 212. As shown in FIG. 3, the next-generation small population generation unit 212 includes a crossover unit 2121, a point mutation unit 2122, and a next-generation small population termination determination unit 2123. The crossover unit 2121 extracts a crossover rate, which is the probability of crossover between individuals, from the parameters set by the parameter setting unit 211, and generates new individuals by crossing two individuals together in accordance with the crossover rate. Note that as the crossover rate increases, the diversity of the individuals is lost, and if the crossover rate is too low, convergence to the optimal individual may be delayed.

[0029] The point mutation unit 2122 extracts a mutation rate, which is the probability that an individual will mutate, from the parameters set by the parameter setting unit 211, and generates new individuals by changing the genetic information of the individuals according to the mutation rate. Note that as the mutation rate increases, the diversity of the individuals is lost, and if the mutation rate is too low, convergence to the optimal individual may be delayed. Individual generation by the crossover unit 2121 and the point mutation unit 2122 may be performed individually or in combination.

[0030] The next-generation small population termination determination unit 2123 determines the termination of generation of one small population by the crossover unit 2121 and the point mutation unit 2122. For example, the next-generation small population termination determination unit 2123 determines that generation of the next-generation small population is terminated when the number of individuals generated by the crossover unit 2121 and the point mutation unit 2122 reaches the number of individuals set for each next-generation small population, which is included in the parameters set by the parameter setting unit 211. If the number of individuals set for each next-generation small population has not been reached and termination is not determined, the crossover unit 2121 and the point mutation unit 2122 repeat individual generation using the parameters set by the parameter setting unit 211.

[0031] The individual storage unit 213 is a storage unit that stores the population of individuals that are the next-generation subpopulations generated by the next-generation subpopulation generation unit 212. Note that the individual storage unit 213 only needs to be accessible by the next-generation subpopulation generation unit 212 and the next-generation subpopulation termination determination unit 2123, and may be provided outside the parameter optimization device 1.

[0032] The evaluation unit 214 is an evaluation unit that evaluates the next-generation small population generated by the next-generation small population generation unit 212 and outputs the evaluation results to the optimization unit 3. For example, the evaluation unit 214 evaluates the individuals of the next-generation small population using fitness, which indicates how suited an individual is to the problem to be solved. When the next-generation small population is composed of N individuals, the evaluation unit 214 calculates the average value of the N fitnesses and outputs this average value to the optimization unit 3 as evaluation information for the next-generation small population. The evaluation unit 214 may also evaluate the next-generation small population using an objective function related to the target problem. The objective function is a function that numerically represents the likelihood of an individual being a solution to the target problem. The evaluation unit 214 evaluates the individuals using the objective function.

[0033] The generation termination determination unit 215 determines the termination of generation of next-generation small populations in the next-generation generation unit 21. For example, the generation termination determination unit 215 determines that generation of next-generation small populations is terminated when the number of next-generation small populations generated by the next-generation generation unit 21 reaches the number of next-generation small populations to be generated that is set for each generation, which is included in the parameters set by the parameter setting unit 211. On the other hand, if the number of next-generation small populations to be generated that is set for each generation has not been reached and termination is not determined, the next-generation generation unit 21 repeats generation of next-generation small populations using the parameters set by the parameter setting unit 211.

[0034] The optimization unit 3 learns how to generate the next-generation subpopulation based on the evaluation results of the next-generation subpopulation, and selects parameters to be used in generating the next-generation subpopulation based on this learning result. For example, the optimization unit 3 performs reinforcement learning such as deep reinforcement learning, Q-learning, or multi-armed bandit. The optimization unit 3 also includes a control evaluation unit 31, an evaluation value storage unit 32, and a selection unit 33, as shown in FIG. 2 .

[0035] The control evaluation unit 31 learns how to generate the next-generation small population using parameters set in the next-generation generation unit 21 and evaluation information on the next-generation small population obtained from the next-generation generation unit 21, and calculates a control evaluation value for the generation of the next-generation small population based on the learning results. The control evaluation value calculated by the control evaluation unit 31 is stored in the evaluation value storage unit 32. As shown in FIG. 2 , the control evaluation unit 31 also includes a control learning unit 311 and a reward generation unit 312.

[0036] The control learning unit 311 performs reinforcement learning to generate the next-generation subpopulation using the reward and the parameters. For example, the control learning unit 311 executes a multi-armed bandit and calculates, from among multiple options, the value of a parameter option that is expected to generate the next-generation subpopulation that is optimal as a solution to the target problem. The value of the option is stored in the evaluation value storage unit 32 as a control evaluation value for generating the next-generation subpopulation.

[0037] For example, the control learning unit 311 calculates the value of the ith option (hereinafter referred to as option i) according to the following formula (1). The most recent reward for option i is the reward generated by the reward generation unit 312 based on the evaluation value of the next generation small group. The most recent observation count for option i is the number of times the parameters related to option i have been observed up to the previous generation. SUM( ) is an operator that finds the sum in ( ). Value of option i = (0.5 + SUM(most recent reward for option i)) / (SUM(most recent observation count for option i) + 1) (1)

[0038] The reward generation unit 312 calculates a reward for the evaluation of the next-generation subpopulation based on the evaluation information. The reward is, for example, the learning progress of the generation of the next-generation subpopulation. Here, the learning progress is a numerical value representing the progress of learning related to the generation of the optimal next-generation subpopulation, such as the fitness of the next-generation subpopulation. The reward may be the learning progress value itself, an evaluation value based on the difference in learning progress before and after the generation change, or an evaluation value based on the magnitude relationship of the learning progress before and after the generation change. By using these rewards, the parameter optimization device 1 can improve the optimality of the parameter selection of the evolutionary algorithm.

[0039] For example, as shown in the following formula (2), if the evaluation value of the next-generation small population based on option i acquired from the evaluation unit 214 is greater than the average of the most recent N evaluation values ​​(fitness) of option i, the reward generation unit 312 sets reward to 1. Otherwise, the reward is set to 0. The reward generated by the reward generation unit 312 is output to the control learning unit 311. Average (most recent N evaluations of option i) < Evaluation of the next-generation small population based on option i (2)

[0040] The evaluation value storage unit 32 is a storage unit that stores the control evaluation value calculated by the control evaluation unit 31. Note that the evaluation value storage unit 32 may be provided outside the parameter optimization device 1 as long as it is accessible by the selection unit 33 and the control learning unit 311.

[0041] The selection unit 33 selects parameters to be set in the evolutionary algorithm unit 2 as control details (hereinafter referred to as "actions") related to the generation of the next-generation small population. For parameter selection, various methods are used, such as the softmax function, the ε-greedy method, the UCB1 algorithm, UCB1-tuned, PM, or AP.

[0042] The selection unit 33 selects parameters to be used in generating the next-generation small population based on the control evaluation values ​​(values ​​of options) stored in the evaluation value storage unit 32. For example, the selection unit 33 normalizes the values ​​of options, which are control evaluation values, according to the following formula (3). The following formula (3) is set so that the normalized values ​​sum up to 1. Next, the selection unit 33 selects options related to parameters related to actions from the options indicated by the normalized values, according to the following formula (4). Normalized value = Value of option / SUM (value of option) (3) Action = Any measure (normalized value) (4)

[0043] Next, a parameter optimization method according to the first embodiment will be described. The parameter optimization method according to the first embodiment includes an evolutionary algorithm step and an optimization step. In the evolutionary algorithm step, the evolutionary algorithm unit 2 generates next-generation subpopulations, which are a group of next-generation individuals with a number smaller than the number of individuals set for each generation of the target problem, using parameters of the evolutionary algorithm set for each generation of the next-generation subpopulation, and evaluates the generated next-generation subpopulations. In the optimization step, the optimization unit 3 learns how to generate next-generation subpopulations based on the evaluation results of the next-generation subpopulations, and selects parameters to be used in generating the next-generation subpopulations based on the learning results. By executing this method, the parameter optimization device 1 can suppress a decrease in the optimality of parameter selection for the evolutionary algorithm.

[0044] 4 is a flowchart showing the parameter optimization method according to the first embodiment, illustrating the detailed flow of the parameter optimization method described above. Of the processes from step ST1 to step ST6 in FIG. 4, the series of processes excluding step ST4-1 and step ST4-2 are the evolutionary algorithm steps. Furthermore, step ST4-1 and step ST4-2 are the optimization steps.

[0045] First, the next-generation small population generation unit 212 randomly generates a next-generation small population as an initial population (step ST1). The parameter setting unit 211 sets the parameters selected by the optimization unit 3 in the next-generation small population generation unit 212 (step ST2). The next-generation small population generation unit 212 generates a new next-generation small population based on the initial population using the parameters set by the parameter setting unit 211 (step ST3).

[0046] The evaluation unit 214 evaluates the next-generation small population generated by the next-generation small population generation unit 212 and outputs the evaluation information to the optimization unit 3 (step ST4). The reward generation unit 312 calculates a reward for the evaluation of the next-generation small population using the evaluation information obtained from the evaluation unit 214. The control learning unit 311 evaluates learning using the reward and parameters (step ST4-1). The selection unit 33 selects parameters to be set in the next-generation generation unit 21 based on the control evaluation value calculated by the control learning unit 311 (step ST4-2).

[0047] In step ST2, the parameter setting unit 211 sets the parameters selected by the optimization unit 3 in the next-generation small population generation unit 212. In this manner, the processes from step ST2 to step ST4 are repeatedly executed. The generation completion determination unit 215 determines whether or not a specified number of individuals for the next generation, i.e., a specified number of next-generation small populations, have been generated (step ST5). Here, if the number of next-generation small populations generated does not reach the specified number (step ST5; NO), the series of processes from step ST2 to step ST4 is executed.

[0048] When the number of next-generation small populations generated reaches a predetermined number (step ST5; YES), the generation change completion determination unit 22 determines whether or not the process has been completed up to the final generation (step ST6). When the process has not been completed up to the final generation (step ST6; NO), the evolutionary algorithm unit 2 performs a generation change and executes the series of processes from step ST2 to step ST5. When the process has been completed up to the final generation (step ST6; YES), the parameter optimization device 1 ends the process of FIG. 4.

[0049] Next, the effectiveness of the parameter optimization device 1 for an optimization problem will be described. FIG. 5 is a schematic diagram showing an example of the traveling salesman problem. The problem shown in FIG. 5 involves searching for the shortest route from a starting point (start) to a destination point (goal) by visiting 18 cities. FIG. 6 is a graph showing simulation results of optimizing the mutation rate included in the parameters of the evolutionary algorithm for the problem shown in FIG. 5. The results in FIG. 6 are obtained when the parameter optimization device 1 and the parameter optimization device 100 perform individual generation for the problem shown in FIG. 5 for 1500 generations.

[0050] 6 , Amax is the maximum value of the fitness of the mutation rate obtained in the individual generation of each generation by the parameter optimization device 1. Amin is the minimum value of the fitness of the mutation rate obtained in the individual generation of each generation by the parameter optimization device 1. Aave is the average value of the fitness included between the minimum and maximum values ​​of the fitness of the mutation rate obtained in the individual generation of each generation by the parameter optimization device 1.

[0051] Bmax is the maximum value of the fitness of the mutation rate obtained in the generation of individuals of each generation by the parameter optimization device 100. Bmin is the minimum value of the fitness of the mutation rate obtained in the generation of individuals of each generation by the parameter optimization device 100. Bave is the average value of the fitness between the minimum and maximum values ​​of the fitness of the mutation rate obtained in the generation of individuals of each generation by the parameter optimization device 100.

[0052] As shown in Fig. 6, a difference Δave occurs between Aave and Bave at around 500 generations. In this way, in the parameter optimization device 1, the fitness improves at a relatively early stage of the generation. Furthermore, a difference Δmin occurs between Amin and Bmin at around 500 generations. However, in the parameter optimization device 1, the minimum fitness value is low at a relatively early stage of the generation. From this, looking at the overall simulation results, it can be seen that the parameter optimization device 1 is able to perform stable learning.

[0053] Next, we will explain the hardware configuration that realizes the functions of the parameter optimization device 1. The evolutionary algorithm unit 2 and optimization unit 3 included in the parameter optimization device 1 are realized by processing circuits. That is, the parameter optimization device 1 includes processing circuits for executing each of the processes from step ST1 to step ST6 shown in Fig. 4. The processing circuit may be dedicated hardware, or it may be a CPU (Central Processing Unit) that executes a program stored in memory.

[0054] Fig. 7A is a block diagram showing a hardware configuration that realizes the functions of the parameter optimization device 1. Fig. 7B is a block diagram showing a hardware configuration that executes software that realizes the functions of the parameter optimization device 1. In Figs. 7A and 7B, an input interface 200 is an interface that relays data related to the target problem that is output from an external device to the parameter optimization device 1. An output interface 201 is an interface that relays an optimal solution to the target problem that is output from the parameter optimization device 1 to a downstream external device.

[0055] 7A , the processing circuit 202 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof. The functions of the evolutionary algorithm unit 2 and the optimization unit 3 included in the parameter optimization device 1 may be realized by separate processing circuits, or these functions may be realized together by a single processing circuit.

[0056] 7B , the functions of the evolutionary algorithm unit 2 and the optimization unit 3 included in the parameter optimization device 1 are realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in the memory 204.

[0057] The processor 203 reads and executes programs stored in the memory 204, thereby realizing the functions of the evolutionary algorithm unit 2 and the optimization unit 3 included in the parameter optimization device 1. For example, the parameter optimization device 1 includes a memory 204 for storing programs that, when executed by the processor 203, result in the processing of steps ST1 to ST6 shown in FIG. 4 being executed. These programs cause a computer to execute the procedures or methods of the processing performed by the evolutionary algorithm unit 2 and the optimization unit 3. The memory 204 may be a computer-readable storage medium that stores programs for causing a computer to function as the evolutionary algorithm unit 2 and the optimization unit 3.

[0058] The memory 204 may be, for example, a non-volatile or volatile semiconductor memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), or an EEPROM (Electrically-EPROM), a magnetic disk, a flexible disk, an optical disk, a compact disk, a minidisk, or a DVD.

[0059] Some of the functions of the evolutionary algorithm unit 2 and the optimization unit 3 included in the parameter optimization device 1 may be realized by dedicated hardware, and the remaining functions may be realized by software or firmware. For example, the evolutionary algorithm unit 2 has its functions realized by a processing circuit 202, which is dedicated hardware, and the optimization unit 3 has its functions realized by a processor 203 reading and executing a program stored in a memory 204. In this way, the processing circuit can realize the above functions by hardware, software, firmware, or a combination of these.

[0060] As described above, the parameter optimization device 1 according to the first embodiment includes an evolutionary algorithm unit 2 that generates next-generation subpopulations, which are populations of the next generation that are smaller in number than the number of individuals set for each generation of the target problem, using parameters of the evolutionary algorithm that are set for each generation of the next-generation subpopulations, and evaluates the next-generation subpopulations; and an optimization unit 3 that learns how to generate next-generation subpopulations based on the evaluation results of the next-generation subpopulations and selects parameters to be used in generating the next-generation subpopulations based on the learning results. The parameter selection and evaluation of the next-generation subpopulations generated using the parameters are performed for each generation of the next-generation subpopulations, and the evaluation results of the next-generation subpopulations are fed back into the subpopulation generation process. This improves adaptability to environmental changes for each generation, enabling the parameter optimization device 1 to improve the optimality of parameter selection for the evolutionary algorithm.

[0061] In the parameter optimization device 1 according to the first embodiment, the evolutionary algorithm unit 2 includes a next-generation small population generation unit 212 that generates a next-generation small population using parameters selected by the optimization unit 3, and an evaluation unit 214 that evaluates the next-generation small population and outputs the evaluation results to the optimization unit 3. The processes performed by the next-generation small population generation unit 212 and the evaluation unit 214 are repeatedly executed until a termination condition for each generation is satisfied, and a new generation occurs when the termination condition is satisfied. This enables the parameter optimization device 1 to improve the optimality of parameter selection for the evolutionary algorithm.

[0062] In the parameter optimization device 1 according to the first embodiment, the optimization unit 3 includes a reward generation unit 312 that calculates a reward for the evaluation of the next-generation small population based on the evaluation information, a control learning unit 311 that performs reinforcement learning to generate the next-generation small population using the reward and the parameters, and a selection unit 33 that selects parameters to be set in the evolutionary algorithm unit 2 as actions for generating the next-generation small population. This enables the parameter optimization device 1 to improve the optimality of parameter selection for the evolutionary algorithm.

[0063] In the parameter optimization device 1 according to the first embodiment, the reward is the learning progress of the generation of the next-generation subpopulation. The parameters are parameter values ​​including a crossover rate and a mutation rate, and operators including a crossover operator and a mutation operator. This enables the parameter optimization device 1 to improve the optimality of parameter selection for the evolutionary algorithm.

[0064] In the parameter optimization device 1 according to the first embodiment, the learning progress is the fitness of the next-generation small population in the evolutionary computation algorithm. The reward is the learning progress value itself, an evaluation value based on the difference in the learning progress before and after the change of generations, or an evaluation value based on the magnitude relationship between the learning progress before and after the change of generations. This enables the parameter optimization device 1 to improve the optimality of parameter selection for the evolutionary algorithm.

[0065] The parameter optimization method according to the first embodiment includes the steps of: an evolutionary algorithm unit 2 generating next-generation subpopulations, which are a group of next-generation individuals with a number smaller than the number of individuals set for each generation of the target problem, using parameters of the evolutionary algorithm set for each generation of the next-generation subpopulation, and evaluating the generated next-generation subpopulations; and an optimization unit 3 learning to generate next-generation subpopulations based on the evaluation results of the next-generation subpopulations and selecting parameters to be used for generating the next-generation subpopulations based on the learning results. By performing this method, the parameter optimization device 1 can improve the optimality of parameter selection for the evolutionary algorithm.

[0066] Any of the components of the embodiments may be modified or omitted.

[0067] The parameter optimization device according to the present disclosure can be used for, for example, various optimization problems.

[0068] 1,100 Parameter optimization device, 2,101 Evolutionary algorithm unit, 3,102 Optimization unit, 21 Next generation generation unit, 22 Generation change termination determination unit, 31 Control evaluation unit, 32 Evaluation value memory unit, 33 Selection unit, 211 Parameter setting unit, 212 Next generation small population generation unit, 213 Individual memory unit, 214 Evaluation unit, 215 Generation termination determination unit, 311 Control learning unit, 312 Reward generation unit, 2121 Crossover unit, 2122 Point mutation unit, 2123 Next generation small population termination determination unit.

Claims

1. A parameter optimization device comprising: an evolutionary algorithm unit that generates subpopulations, which are next-generation populations whose number is smaller than the number of individuals set for each generation of a target problem, using parameters of an evolutionary algorithm that are set for each generation of the subpopulations, and evaluates the generated subpopulations; and an optimization unit that learns how to generate the subpopulations based on the results of the evaluation of the subpopulations, and selects the parameters used by the evolutionary algorithm unit to generate the subpopulations based on the learning results.

2. The parameter optimization device described in claim 1, characterized in that the evolutionary algorithm unit comprises: a generation unit that generates the small population using the parameters selected by the optimization unit; and an evaluation unit that evaluates the small population and outputs the evaluation result to the optimization unit, and each process by the generation unit and the evaluation unit is repeatedly executed until a termination condition for each generation is satisfied, and a new generation is initiated when the termination condition is satisfied.

3. The parameter optimization device of claim 2, characterized in that the optimization unit comprises: a reward generation unit that calculates a reward for the evaluation of the small population based on the evaluation result; a control learning unit that performs reinforcement learning for the generation of the small population using the reward and the parameters; and a selection unit that selects the parameters to be set in the evolutionary algorithm unit as actions for generating the small population.

4. The parameter optimization device described in claim 3, characterized in that the reward is the learning progress of the generation of the small population, and the parameters are parameter values ​​including a crossover rate and a mutation rate, and operators including a crossover operator and a mutation operator.

5. The parameter optimization device according to claim 4, characterized in that the learning progress is the fitness of the small population in an evolutionary computation algorithm, and the reward is the value of the learning progress itself, an evaluation value based on the difference between the learning progress before and after the change of generations, or an evaluation value based on the relationship between the magnitude of the learning progress before and after the change of generations.

6. A parameter optimization method using a parameter optimization device, comprising: a step in which an evolutionary algorithm unit generates small populations, which are a next-generation population group whose number is smaller than the number of individuals set for each generation of the target problem, using parameters of an evolutionary algorithm that are set for each generation of the small population, and evaluates the generated small populations; and a step in which an optimization unit learns to generate the small populations based on the results of the evaluation of the small populations, and selects the parameters to be used by the evolutionary algorithm unit to generate the small populations based on the learning results.

Images