Parameter optimization device and parameter optimization method

Abstract

The parameter optimization device (1) includes an evolutionary algorithm unit (2) that generates a next-generation subpopulation, which is a next-generation population of individuals fewer than the number of individuals set for each generation of the target problem, using parameters of an evolutionary algorithm that are set for each generation of the next-generation subpopulation, and evaluates the next-generation subpopulation, and an optimization unit (3) that learns to generate subpopulations based on the evaluation results of the next-generation subpopulation, and selects the parameters to be used in generating the next-generation subpopulation based on the learning results.

Description

[Technical field]

[0001] The present disclosure relates to a parameter optimization device and a parameter optimization method. [Background technology]

[0002] Evolutionary algorithms are widely used as a method for solving various optimization problems. For example, the learning method described in Patent Document 1 learns a strategy (π) that optimally adapts at least one parameter (σ) of an evolutionary algorithm, in particular a CMA-ES (covariance matrix adaptive evolution strategy) algorithm or a differential evolutionary algorithm. The method includes initializing a strategy for computing a parameter representation (A) of a parameter (σ) depending on state information (S) about a problem instance, and learning a strategy (π) using reinforcement learning, where which parameter representation is optimal for possible state information is learned based on an interaction between the parameter representation determined using the strategy depending on the state information (S), the problem instance, and a reward signal (R). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-21177 Summary of the Invention [Problem to be solved by the invention]

[0004] In parameter optimization using conventional evolutionary algorithms, parameters are selected for each generation, and the selected parameters are used to generate a set number of individuals (candidate solutions) for each generation, and then the generated population of individuals is evaluated collectively. Therefore, when there is large variation in the environment over one generation, the evaluation of individuals over one generation cannot keep up with the environmental variations, resulting in a decrease in the optimality of parameter selection. For example, changes in the environment during one generation may cause the constraints of the problem to be solved by the evolutionary algorithm to fluctuate or the objective function to change. Also, the timing or pattern of evolution may change. These changes in environmental factors have a significant impact on parameter selection.

[0005] In addition, in the learning method described in Patent Document 1, the selection and learning of the parameter (σ) of the evolutionary algorithm is performed for each generation. Therefore, when the environment changes significantly in one generation, the learning method described in Patent Document 1 is likely to reduce the optimality of the parameter selection and reduce the learning efficiency of the evolutionary algorithm.

[0006] The present disclosure is devised to solve the above-mentioned problems, and aims to provide a parameter optimization device that can improve the optimality of parameter selection for an evolutionary algorithm. [Means for solving the problem]

[0007] The parameter optimization device according to the present disclosure includes an evolutionary algorithm unit that generates subpopulations, which are a next-generation population group with a number of individuals less than the number of individuals set for each generation of a target problem, using parameters of an evolutionary algorithm that are set for each generation of the subpopulations, and evaluates the generated subpopulations; and Learning Based on the learning results, 、 and an optimization unit that selects parameters used by the evolutionary algorithm unit to generate the small population. Effect of the Invention

[0008] According to the present disclosure, subpopulations, which are next-generation populations with a number smaller than the number of individuals set for each generation of the target problem, are generated using parameters of an evolutionary algorithm set for each generation of the subpopulations, learning is performed for generating the subpopulations based on evaluation results of the generated subpopulations, and parameters to be used for generating the subpopulations are selected based on the learning results. By selecting parameters and evaluating the subpopulations generated using the parameters for each generation of a subpopulation, the evaluation results of the subpopulations for the next generation are fed back to the process of generating the subpopulations. This improves adaptability to environmental changes for each generation, and the parameter optimization device according to the present disclosure can increase the optimality of parameter selection for the evolutionary algorithm. [Brief description of the drawings]

[0009] [Figure 1] 1A and 1B are schematic diagrams showing an overview of parameter optimization of an evolutionary algorithm. [Diagram 2] 1 is a block diagram showing an example of the configuration of a parameter optimization device according to a first embodiment. [Diagram 3] 13 is a block diagram showing an example of the configuration of a next-generation small group generation unit. FIG. [Figure 4] 3 is a flowchart showing a parameter optimization method according to the first embodiment. [Diagram 5] FIG. 1 is a schematic diagram illustrating an example of a traveling salesman problem. [Figure 6] 6 is a graph showing the results of a simulation of optimizing the mutation rate included in the parameters of the evolutionary algorithm for the problem of FIG. 5. [Figure 7] 7A and 7B are block diagrams showing a hardware configuration for realizing the functions of the parameter optimization device according to the first embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Embodiment 1 First, problems in parameter optimization of a conventional evolutionary algorithm will be described with reference to Fig. 1A. Fig. 1A is a schematic diagram showing an overview of parameter optimization of an evolutionary algorithm, and shows parameter optimization by a conventional parameter optimization device 100. The parameter optimization device 100 includes an evolutionary algorithm unit 101 and an optimization unit 102. The evolutionary algorithm unit 101 generates a subpopulation, which is a next-generation population of the number of individuals set for each generation of a target problem, by using parameters of an evolutionary algorithm set for each generation, and evaluates the generated subpopulation.

[0011] Evolutionary algorithms obtain optimal solutions by evolving a population of individuals that are candidate solutions to a target problem through repeated generational changes based on changes and selection. Evolutionary algorithms have the ability to deal with various optimization problems. However, the performance of evolutionary algorithms depends on parameter tuning, i.e., whether the optimal values ​​of hyperparameters such as crossover rate and mutation rate, and operators such as crossover operator and mutation operator can be selected.

[0012] In order to improve the optimality of parameter selection, AOS (Adaptive Operator Selection) has been proposed in the past, which solves problems by applying a multi-armed bandit as the evolutionary algorithm unit 101. The parameter optimization device 100 performs AOS. While learning the parameters of the evolutionary algorithm, AOS evaluates the individuals generated in the current generation using only the history information of the individual generation executed up to the current generation, and determines the individuals to be left in the next generation based on the evaluation results. Since the optimal parameters for each generation are selected by AOS, it becomes possible to efficiently control the optimal individual generation.

[0013] For example, in a generation update task T (processing of generation T), the optimization unit 102 calculates a fitness f T and obtain the fitness f TBased on this, the optimal parameters (e.g., the crossover rate p T and the mutation rate e T The optimization unit 102 sets the selected parameters in the evolutionary algorithm unit 101. For example, the fitness f T is an index showing how well an individual meets the objective function. T The higher the individual, the higher the probability that it will be passed on to the next generation.

[0014] The evolutionary algorithm unit 101 performs crossover and point mutation using the parameters of generation T set by the optimization unit 102 to generate a small population of individuals of the next generation T+1. Crossover is an operation of combining the genetic information of two previous generation individuals (parent individuals) to generate new next generation individuals (descendant individuals). The evolutionary algorithm unit 101 generates individuals of the next generation T+1 by crossover based on the crossover rate set by the optimization unit 102. Point mutation is an operation of randomly changing part of the genetic information of an individual. The evolutionary algorithm unit 101 generates individuals of the next generation T+1 by point mutation based on the mutation rate set by the optimization unit 102.

[0015] The evolutionary algorithm unit 101 generates a small population of individuals of the next generation T+1 consisting of a group of individuals of the number of candidate solutions included in the parameters of the generation T, and performs an evaluation process (fitness evaluation) of this small population to obtain a fitness f T+1 The evolutionary algorithm unit 101 then switches from generation T to generation T+1, and in the generation update operation T+1 (processing of generation T+1), generates a small population of individuals of the next generation T+2 consisting of the population of candidate solutions included in the parameters of generation T+1. The evolutionary algorithm unit 101 generates a small population of individuals of the next generation T+2, and calculates the fitness f T+2 to the optimization unit 102. This series of processes is repeated until a preset number of generations is reached.

[0016] As shown in FIG. 1A, in a conventional parameter optimization device 100, parameters are selected for each generation, and information up to the previous generation (the fitness f T) is used to generate a small population for the next generation consisting of the number of individuals set for that generation, and the generated small populations are evaluated together. For this reason, if there is a large change in the environment in one generation, the evaluation of individuals in one generation cannot keep up with the change in the environment, and the optimality of parameter selection decreases. In this way, AOS selects and evaluates parameters collectively for each generation, which means that parameters are always selected based only on the evaluation of the previous generation, which is inefficient.

[0017] In contrast to this, the parameter optimization device according to the first embodiment selects parameters and evaluates small populations generated using the parameters each time a small population is generated. 1B is a schematic diagram showing an overview of parameter optimization of an evolutionary algorithm by a parameter optimization device 1 according to embodiment 1. The parameter optimization device 1 includes an evolutionary algorithm unit 2 and an optimization unit 3. The evolutionary algorithm unit 2 generates small populations, which are next-generation populations whose number is smaller than the number of individuals set for each generation of a target problem, by using parameters of the evolutionary algorithm that are set for each generation of a small population, and evaluates the generated small populations.

[0018] In the generation update task T, the optimization unit 3 calculates the fitness f of a small group of individuals k of the next generation T+1 generated sequentially by the evolutionary algorithm unit 2. Tk Then, the optimization unit 3 optimizes the parameters (e.g., crossover rate p Tk+1 and the mutation rate e Tk+1 ) and set the selected parameters in the evolutionary algorithm section 2.

[0019] The evolutionary algorithm unit 2 generates a small population of individuals k of the next generation T+1 using the parameters set by the optimization unit 3, and performs an evaluation process (fitness evaluation) of this small population to determine the fitness f TkThe evolutionary algorithm unit 2 then switches from generation T to generation T+1, and in the generation update operation T+1, sequentially generates a small population of individuals for the next generation T+2, which is made up of a population with a number of individuals less than the number of candidate solutions included in the parameters of the generation T+1. The evolutionary algorithm unit 2 then calculates the fitness f T+1k is output to the optimization unit 3.

[0020] The optimization unit 3 learns how to generate a subpopulation, and based on the results of this learning, optimizes parameters (e.g., crossover rate p T+1k+1 and the mutation rate e T+1k+1 ) and set it to Evolutionary Algorithm Section 2. This series of processes is repeated until a preset number of individuals for the next generation is reached. In this way, in the parameter optimization device 1, the evaluation results of the next generation of small populations are fed back into the process of generating small populations, thereby improving adaptability to environmental changes in each generation and making it possible to improve the optimality of parameter selection in the evolutionary algorithm.

[0021] Fig. 2 is a block diagram showing an example of the configuration of the parameter optimization device 1. As shown in Fig. 2, the parameter optimization device 1 is a device equipped with a calculation unit and a storage unit. The calculation unit controls the overall operation of the parameter optimization device 1. The calculation unit includes an evolutionary algorithm unit 2 and an optimization unit 3. The calculation unit executes an information processing application for parameter optimization, thereby realizing various functions of the evolutionary algorithm unit 2 and the optimization unit 3.

[0022] The storage unit stores an information processing application for parameter optimization and information used in the calculation process of the calculation unit. This storage unit realizes, for example, the evaluation value storage unit 32 and the individual storage unit 213. The storage unit may be a storage device included in a computer functioning as the parameter optimization device 1. For example, the storage unit may include a storage such as a hard disk drive (HDD) or a solid state drive (SSD), or the memory 104 in FIG. 7B described later. The storage unit may be provided outside the parameter optimization device 1 as long as it is accessible by the parameter optimization device 1.

[0023] The evolutionary algorithm unit 2 generates a next-generation subpopulation, which is a next-generation population of individuals fewer in number than the number of individuals set for each generation of the target problem, using parameters of the evolutionary algorithm set for each generation of the next-generation subpopulation, and evaluates the next-generation subpopulation. For example, the evolutionary algorithm unit 2 generates individuals for the target problem by repeating N generation changes from the initial generation, and generates individuals of the final generation with improved fitness. Note that N is an integer of 2 or more.

[0024] The evolutionary algorithm operated in the evolutionary algorithm unit 2 is, for example, a genetic algorithm, which is an algorithm for performing generation alternation, a differential evolution algorithm, a multi-objective optimization algorithm, etc. As shown in FIG. 2, the evolutionary algorithm unit 2 includes a next-generation generation unit 21 and a generation alternation end determination unit 22.

[0025] The next generation generation unit 21 generates a next generation small population for a target problem using an evolutionary algorithm and evaluates the generated population, and includes a parameter setting unit 211, a next generation small population generation unit 212, an individual storage unit 213, an evaluation unit 214, and a generation completion determination unit 215. In addition, the generation change completion determination unit 22 determines whether or not the next generation generation unit 21 has performed operations from the initial generation to the final generation set in the next generation generation unit 21. For example, when the final generation is the Nth generation, the next generation generation unit 21 generates final generation individuals with improved fitness by repeating N generation changes from the initial generation until generating individuals of the Nth generation.

[0026] The parameter setting unit 211 sets the parameters acquired from the optimization unit 3 in the next-generation small population generation unit 212. The parameters acquired from the optimization unit 3 are so-called hyperparameters, and are parameters that need to be adjusted when operating the evolutionary algorithm. For example, the parameters include the number of generations for identifying the final generation in which the next-generation generation unit 21 generates individuals, the number of individuals generated by the next-generation small population generation unit 212, the crossover rate, the mutation rate, and a method for selecting individuals to be used for generating individuals in the next generation. In addition, the number of individuals included in the parameters includes not only the number of individuals set for each generation, as in conventional evolutionary algorithms, but also the number of next-generation small populations generated by the next-generation generation unit 21 and the number of individuals for each small population generated by the next-generation small population generation unit 212.

[0027] The next-generation small population generation unit 212 is a generation unit that generates a next-generation small population using the parameters selected by the optimization unit 3. The next-generation small population is a small population consisting of a plurality of individuals that are candidate solutions to the target problem. The next-generation small population includes the number of individuals set for each next-generation small population that are included in the parameters selected by the optimization unit 3. Note that the number of individuals for each next-generation small population is assumed to be less than the number of individuals set for that generation.

[0028] Fig. 3 is a block diagram showing an example of the configuration of the next-generation small population generation unit 212. As shown in Fig. 3, the next-generation small population generation unit 212 includes a crossover unit 2121, a point mutation unit 2122, and a next-generation small population end determination unit 2123. The crossover unit 2121 extracts a crossover rate, which is the probability of crossover between individuals, from the parameters set by the parameter setting unit 211, and crosses two individuals according to the crossover rate to generate new individuals. Note that as the crossover rate increases, the diversity of individuals is lost, and if the crossover rate is too low, convergence to an optimal individual may be delayed.

[0029] The point mutation unit 2122 extracts a mutation rate, which is the probability that an individual will mutate, from the parameters set by the parameter setting unit 211, and generates new individuals by changing the genetic information of the individuals according to the mutation rate. Note that as the mutation rate increases, the diversity of the individuals is lost, and if the mutation rate is too low, convergence to the optimal individual may be delayed. The generation of individuals by the crossover section 2121 and the point mutation section 2122 may be performed separately or in combination.

[0030] The next-generation small population end determination unit 2123 determines the end of generation of one small population by the crossover unit 2121 and the point mutation unit 2122. For example, the next-generation small population end determination unit 2123 determines the end of generation of the next-generation small population when the number of individuals generated by the crossover unit 2121 and the point mutation unit 2122 reaches the number of individuals set for each next-generation small population, which is included in the parameters set by the parameter setting unit 211. If the number of individuals set for each next-generation small population has not been reached and the end determination is not made, the crossover unit 2121 and the point mutation unit 2122 repeat the generation of individuals using the parameters set by the parameter setting unit 211.

[0031] The individual storage unit 213 is a storage unit that stores the individuals that are the next-generation small population generated by the next-generation small population generation unit 212. Note that the individual storage unit 213 only needs to be accessible by the next-generation small population generation unit 212 and the next-generation small population termination determination unit 2123, and may be provided outside the parameter optimization device 1.

[0032] The evaluation unit 214 is an evaluation unit that evaluates the next-generation small population generated by the next-generation small population generation unit 212 and outputs the evaluation result to the optimization unit 3. For example, the evaluation unit 214 evaluates the individuals of the next-generation small population using fitness that indicates how suitable an individual is for the problem to be solved. When the next-generation small population is composed of N individuals, the evaluation unit 214 calculates the average value of the N fitness values ​​and outputs this average value to the optimization unit 3 as evaluation information of the next-generation small population. The evaluation unit 214 may also evaluate the next-generation small population using an objective function related to the target problem. The objective function is a function that numerically represents the likelihood of a solution to the target problem. The evaluation unit 214 evaluates the individuals using the objective function.

[0033] The generation end determination unit 215 determines the end of generation of the next generation small population in the next generation generation unit 21. For example, the generation end determination unit 215 determines that the generation of the next generation small population is ended when the number of next generation small populations generated by the next generation generation unit 21 reaches the number of next generation small populations to be generated set for each generation, which is included in the parameters set by the parameter setting unit 211. On the other hand, if the number of next generation small populations to be generated set for each generation has not been reached and an end determination is not made, the next generation generation unit 21 repeats generation of the next generation small population using the parameters set by the parameter setting unit 211.

[0034] The optimization unit 3 learns how to generate a next-generation small population based on the evaluation result of the next-generation small population, and selects parameters to be used in generating the next-generation small population based on the learning result. For example, the optimization unit 3 performs reinforcement learning such as deep reinforcement learning, Q-learning, or multi-armed bandit. The optimization unit 3 also includes a control evaluation unit 31, an evaluation value storage unit 32, and a selection unit 33, as shown in FIG. 2.

[0035] The control evaluation unit 31 learns the generation of the next-generation small population using parameters set in the next-generation generation unit 21 and evaluation information of the next-generation small population acquired from the next-generation generation unit 21, and calculates a control evaluation value related to the generation of the next-generation small population based on the learning result. The control evaluation value calculated by the control evaluation unit 31 is stored in the evaluation value storage unit 32. Furthermore, the control evaluation unit 31 includes a control learning unit 311 and a reward generation unit 312, as shown in FIG. 2.

[0036] The control learning unit 311 performs reinforcement learning for the generation of the next-generation small population using the reward and the parameters. For example, the control learning unit 311 executes a multi-armed bandit and calculates the value of a parameter option that is expected to generate an optimal next-generation small population as a solution to the target problem from among multiple options. The value of the option is stored in the evaluation value storage unit 32 as a control evaluation value for the generation of the next-generation small population.

[0037] For example, the control learning unit 311 calculates the value of the i-th option (hereinafter referred to as option i) according to the following formula (1). The most recent reward for option i is the reward generated by the reward generation unit 312 based on the evaluation value of the next-generation small population. The most recent observation count for option i is the number of times the parameters related to option i have been observed up to the previous generation. SUM( ) is an operator that finds the sum in ( ). Value of choice i = (0.5 + SUM(recent reward of choice i)) / (SUM(recent number of observations of choice i) + 1) (1)

[0038] The reward generation unit 312 calculates a reward for the evaluation of the next-generation small population based on the evaluation information. The reward is, for example, the learning progress of the generation of the next-generation small population. Here, the learning progress is a numerical value indicating the progress of learning regarding the generation of the optimal next-generation small population, for example, the fitness of the next-generation small population. The reward may be the value of the learning progress itself, an evaluation value based on the difference in the learning progress before and after the generation change, or an evaluation value based on the magnitude relationship of the learning progress before and after the generation change. By using these rewards, the parameter optimization device 1 can improve the optimality of the parameter selection of the evolutionary algorithm.

[0039] For example, as shown in the following formula (2), if the evaluation value of the next generation small population based on option i acquired from the evaluation unit 214 is greater than the average of the most recent N evaluation values ​​(fitness) of option i, the reward generation unit 312 sets reward = 1. Otherwise, the reward is set to 0. The reward generated by the reward generation unit 312 is output to the control learning unit 311. Average (last N evaluations of option i) < Evaluation of next generation small group by option i (2)

[0040] The evaluation value storage unit 32 is a storage unit that stores the control evaluation value calculated by the control evaluation unit 31. Note that the evaluation value storage unit 32 may be provided outside the parameter optimization device 1 as long as it is accessible by the selection unit 33 and the control learning unit 311.

[0041] The selection unit 33 selects parameters to be set in the evolutionary algorithm unit 2 as control details (hereinafter, referred to as "actions") related to the generation of the next-generation small population. For parameter selection, various methods such as the softmax function, the ε-greedy method, the UCB1 algorithm, UCB1-tuned, PM, or AP are used.

[0042] The selection unit 33 selects parameters to be used in generating the next-generation small population based on the control evaluation value (value of the option) stored in the evaluation value storage unit 32. For example, the selection unit 33 normalizes the value of the option, which is the control evaluation value, according to the following formula (3). The following formula (3) is set so that the normalized values ​​sum up to 1. Next, the selection unit 33 selects an option for the parameter related to the behavior from the options indicated by the normalized values ​​in accordance with the following formula (4). Normalized Value = Choice Value / SUM(Choice Value) (3) Action = any policy (normalized value) (4)

[0043] Next, the parameter optimization method according to the first embodiment will be described. The parameter optimization method according to the first embodiment includes an evolutionary algorithm step and an optimization step. In the evolutionary algorithm step, the evolutionary algorithm unit 2 generates a next-generation subpopulation, which is a next-generation population group having a number smaller than the number of individuals set for each generation of the target problem, using parameters of the evolutionary algorithm set for each generation of the next-generation subpopulation, and evaluates the generated next-generation subpopulation. In the optimization step, the optimization unit 3 learns the generation of the next-generation subpopulation based on the evaluation result of the next-generation subpopulation, and selects parameters to be used for generating the next-generation subpopulation based on the learning result. The parameter optimization device 1 executes this method, thereby making it possible to suppress a decrease in optimality of parameter selection of the evolutionary algorithm.

[0044] Fig. 4 is a flowchart showing the parameter optimization method according to the first embodiment, and shows a detailed flow of the parameter optimization method described above. Among the processes from step ST1 to step ST6 in Fig. 4, a series of processes excluding step ST4-1 and step ST4-2 are evolutionary algorithm steps. Furthermore, step ST4-1 and step ST4-2 are optimization steps.

[0045] First, the next-generation small population generating unit 212 randomly generates a next-generation small population as an initial population (step ST1). The parameter setting unit 211 sets the parameters selected by the optimization unit 3 in the next-generation small population generating unit 212 (step ST2). The next-generation small population generating unit 212 uses the parameters set by the parameter setting unit 211 to generate a new next-generation small population based on the initial population (step ST3).

[0046] The evaluation unit 214 evaluates the next-generation small population generated by the next-generation small population generation unit 212, and outputs evaluation information to the optimization unit 3 (step ST4). The reward generation unit 312 calculates a reward for the evaluation of the next-generation small population using the evaluation information acquired from the evaluation unit 214. The control learning unit 311 evaluates the learning using the reward and the parameters (step ST4-1). The selection unit 33 selects parameters to be set in the next generation generation unit 21 based on the control evaluation value calculated by the control learning unit 311 (step ST4-2).

[0047] In step ST2, the parameter setting unit 211 sets the parameters selected by the optimization unit 3 in the next-generation small population generation unit 212. In this manner, the processes from step ST2 to step ST4 are repeatedly executed. The generation end determination unit 215 determines whether or not a prescribed number of individuals for the next generation, i.e., a prescribed number of next-generation small populations, have been generated (step ST5). If the number of next-generation small populations generated does not reach the prescribed number (step ST5; NO), a series of processes from step ST2 to step ST4 are executed.

[0048] When the number of next-generation small populations generated reaches a prescribed number (step ST5; YES), the generation change end determination unit 22 determines whether the process is completed up to the final generation (step ST6). When the process is not completed up to the final generation (step ST6; NO), the evolutionary algorithm unit 2 performs generation change and executes a series of processes from step ST2 to step ST5. When the process is completed up to the final generation (step ST6; YES), the parameter optimization device 1 ends the process of FIG. 4.

[0049] Next, the effectiveness of the parameter optimization device 1 for solving the optimization problem will be described. FIG. 5 is a schematic diagram showing an example of the traveling salesman problem. The problem shown in FIG. 5 is to search for the shortest route from a start point to a goal point by visiting 18 cities. FIG. 6 is a graph showing a simulation result of optimizing the mutation rate included in the parameters of the evolutionary algorithm for the problem shown in FIG. 5. The result in FIG. 6 is obtained when the parameter optimization device 1 and the parameter optimization device 100 perform 1500 generations of individual generation for the problem shown in FIG. 5.

[0050] 6, Amax is the maximum value of the fitness of the mutation rate obtained in the generation of individuals of each generation by the parameter optimization device 1. Amin is the minimum value of the fitness of the mutation rate obtained in the generation of individuals of each generation by the parameter optimization device 1. Aave is the average value of the fitness between the minimum and maximum values ​​of the fitness of the mutation rate obtained in the generation of individuals of each generation by the parameter optimization device 1.

[0051] Bmax is the maximum value of the fitness of the mutation rate obtained in the generation of individuals of each generation by the parameter optimization device 100. Bmin is the minimum value of the fitness of the mutation rate obtained in the generation of individuals of each generation by the parameter optimization device 100. Bave is the average value of the fitness between the minimum and maximum values ​​of the fitness of the mutation rate obtained in the generation of individuals of each generation by the parameter optimization device 100.

[0052] As shown in FIG. 6, a difference Δave occurs between Aave and Bave at about 500 generations. In this way, in the parameter optimization device 1, the fitness improves at a relatively early stage of the generation. Also, a difference Δmin occurs between Amin and Bmin at about 500 generations. However, in the parameter optimization device 1, the minimum fitness value is low at a relatively early stage of the generation. From this, looking at the overall simulation results, it can be seen that the parameter optimization device 1 is able to perform stable learning.

[0053] Next, a hardware configuration for realizing the functions of the parameter optimization device 1 will be described. The evolutionary algorithm unit 2 and the optimization unit 3 included in the parameter optimization device 1 are realized by processing circuits. That is, the parameter optimization device 1 includes a processing circuit for executing each process from step ST1 to step ST6 shown in Fig. 4. The processing circuit may be dedicated hardware, or may be a CPU (Central Processing Unit) that executes a program stored in a memory.

[0054] Fig. 7A is a block diagram showing a hardware configuration for realizing the functions of the parameter optimization device 1. Fig. 7B is a block diagram showing a hardware configuration for executing software for realizing the functions of the parameter optimization device 1. In Figs. 7A and 7B, an input interface 200 is an interface that relays data relating to a target problem output from an external device to the parameter optimization device 1. An output interface 201 is an interface that relays an optimal solution to the target problem output from the parameter optimization device 1 to a downstream external device.

[0055] 7A, the processing circuit 202 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination of these. The functions of the evolutionary algorithm unit 2 and the optimization unit 3 included in the parameter optimization device 1 may be realized by separate processing circuits, or these functions may be realized together by one processing circuit.

[0056] 7B, the functions of the evolutionary algorithm unit 2 and the optimization unit 3 included in the parameter optimization device 1 are realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in the memory 204.

[0057] The processor 203 reads out and executes the programs stored in the memory 204, thereby realizing the functions of the evolutionary algorithm unit 2 and the optimization unit 3 included in the parameter optimization device 1. For example, the parameter optimization device 1 includes a memory 204 for storing a program that, when executed by the processor 203, results in the processing of steps ST1 to ST6 shown in FIG. 4 being executed. These programs cause a computer to execute the procedures or methods of the processing performed by the evolutionary algorithm unit 2 and the optimization unit 3. The memory 204 may be a computer-readable storage medium in which a program for causing a computer to function as the evolutionary algorithm unit 2 and the optimization unit 3 is stored.

[0058] Memory 204 may be, for example, a non-volatile or volatile semiconductor memory such as a Random Access Memory (RAM), a Read Only Memory (ROM), a flash memory, an Erasable Programmable Read Only Memory (EPROM), an Electrically-EPROM (EEPROM), a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, a DVD, etc.

[0059] A part of the functions of the evolutionary algorithm unit 2 and the optimization unit 3 included in the parameter optimization device 1 may be realized by dedicated hardware, and the rest may be realized by software or firmware. For example, the evolutionary algorithm unit 2 realizes its functions by a processing circuit 202, which is dedicated hardware, and the optimization unit 3 realizes its functions by a processor 203 reading and executing a program stored in a memory 204. Thus, the processing circuitry may implement the above-described functions through hardware, software, firmware or a combination of these.

[0060] As described above, the parameter optimization device 1 according to the first embodiment includes an evolutionary algorithm unit 2 that generates a next-generation subpopulation, which is a population of next-generation individuals less than the number of individuals set for each generation of the target problem, using parameters of the evolutionary algorithm set for each generation of the next-generation subpopulation, and evaluates the next-generation subpopulation, and an optimization unit 3 that learns the generation of the next-generation subpopulation based on the evaluation result of the next-generation subpopulation, and selects the parameters to be used for generating the next-generation subpopulation based on the learning result. The selection of parameters and the evaluation of the next-generation subpopulation generated using the parameters are performed for each generation of the next-generation subpopulation, and the evaluation result of the next-generation subpopulation is fed back to the subpopulation generation process. This improves adaptability to environmental changes for each generation, and the parameter optimization device 1 can improve the optimality of parameter selection of the evolutionary algorithm.

[0061] In the parameter optimization device 1 according to the first embodiment, the evolutionary algorithm unit 2 includes a next-generation small population generation unit 212 that generates a next-generation small population using parameters selected by the optimization unit 3, and an evaluation unit 214 that evaluates the next-generation small population and outputs the evaluation result to the optimization unit 3. The processes by the next-generation small population generation unit 212 and the evaluation unit 214 are repeatedly executed until a termination condition for each generation is satisfied, and a new generation is initiated when the termination condition is satisfied. This enables the parameter optimization device 1 to improve the optimality of parameter selection for the evolutionary algorithm.

[0062] In the parameter optimization device 1 according to the first embodiment, the optimization unit 3 includes a reward generation unit 312 that calculates a reward for the evaluation of the next-generation small population based on the evaluation information, a control learning unit 311 that performs reinforcement learning for generating the next-generation small population using the reward and parameters, and a selection unit 33 that selects parameters to be set in the evolutionary algorithm unit 2 as actions for generating the next-generation small population. This enables the parameter optimization device 1 to improve the optimality of parameter selection for the evolutionary algorithm.

[0063] In the parameter optimization device 1 according to the first embodiment, the reward is the learning progress of the generation of the next-generation small population. The parameters are parameter values ​​including a crossover rate and a mutation rate, and operators including a crossover operator and a mutation operator. This allows the parameter optimization device 1 to improve the optimality of parameter selection of the evolutionary algorithm.

[0064] In the parameter optimization device 1 according to the first embodiment, the learning progress is the fitness of the next-generation small population in the evolutionary computation algorithm. The reward is the learning progress value itself, an evaluation value based on the difference in the learning progress before and after the generation change, or an evaluation value based on the magnitude relationship between the learning progress before and after the generation change. This enables the parameter optimization device 1 to improve the optimality of parameter selection for the evolutionary algorithm.

[0065] The parameter optimization method according to the first embodiment includes a step in which an evolutionary algorithm unit 2 generates a next-generation subpopulation, which is a next-generation population group having a number of individuals less than the number of individuals set for each generation of the target problem, using parameters of the evolutionary algorithm set for each generation of the next-generation subpopulation, and evaluates the generated next-generation subpopulation, and a step in which an optimization unit 3 learns to generate the next-generation subpopulation based on the evaluation result of the next-generation subpopulation, and selects parameters to be used for generating the next-generation subpopulation based on the learning result. The parameter optimization device 1 executes this method, thereby making it possible to improve the optimality of parameter selection for the evolutionary algorithm.

[0066] Any of the components of the embodiments may be modified or omitted. [Industrial Applicability]

[0067] The parameter optimization device according to the present disclosure can be used for, for example, various optimization problems. [Explanation of symbols]

[0068] 1,100 parameter optimization device, 2,101 evolutionary algorithm unit, 3,102 optimization unit, 21 next generation generation unit, 22 generation change end judgment unit, 31 control evaluation unit, 32 evaluation value memory unit, 33 selection unit, 211 parameter setting unit, 212 next generation small population generation unit, 213 individual memory unit, 214 evaluation unit, 215 generation end judgment unit, 311 control learning unit, 312 reward generation unit, 2121 crossover unit, 2122 point mutation unit, 2123 next generation small population end judgment unit.

Claims

1. An evolutionary algorithm unit that generates a small population, which is a population of the next generation with a number less than the number of individuals set for each generation of the target problem, using parameters of an evolutionary algorithm set for each generation of the small population, and evaluates the generated small population; An optimization unit that learns based on the evaluation result of the small population and selects the parameters used by the evolutionary algorithm unit for generating the small population based on the learning result; and A parameter optimization device characterized by the above.

2. The evolutionary algorithm unit includes: A generation unit that generates the small population using the parameters selected by the optimization unit; An evaluation unit that evaluates the small population and outputs the evaluation result to the optimization unit; and Each process by the generation unit and the evaluation unit is repeatedly executed until an end condition for each generation is satisfied, and when the end condition is satisfied, a generation change occurs. The parameter optimization device according to claim 1, characterized by the above.

3. The optimization unit includes: A reward generation unit that calculates a reward for the evaluation of the small population based on the evaluation result; A control learning unit that performs reinforcement learning on the generation of the small population using the reward and the parameters; and A selection unit that selects the parameters to be set in the evolutionary algorithm unit as an action for generating the small population. The parameter optimization device according to claim 2, characterized by the above.

4. The reward is the learning progress of the generation of the small population, The parameters are parameter values including a crossover rate and a mutation rate, and operators including a crossover operator and a mutation operator. The parameter optimization device according to claim 3, characterized by the above.

5. The learning progress is the fitness of the small population in an evolutionary calculation algorithm, The reward is a value of the learning progress, an evaluation value based on a difference in the learning progress before and after generation change, or an evaluation value based on a magnitude relationship of the learning progress before and after generation change. The parameter optimization device according to claim 4, characterized by the above.

6. A parameter optimization method by a parameter optimization device, comprising: A step in which an evolutionary algorithm unit generates a small population, which is a population of the next generation with a number less than the number of individuals set for each generation of the target problem, using parameters of an evolutionary algorithm set for each generation of the small population, and evaluates the generated small population; A step in which the optimization unit learns based on the evaluation results of the small population and selects, based on the learning results, the parameters used by the evolutionary algorithm unit to generate the small population. A parameter optimization method characterized by the above. **Claim 7** A computer is provided with: A process of generating, for each generation of a target problem, a small population that is a population of individuals in the next generation and has a number smaller than the number of individuals set for each generation, using parameters of an evolutionary algorithm set for each generation of generating the small population, and evaluating the generated small population. A program for causing the computer to execute a process of learning based on the evaluation results of the small population and selecting, based on the learning results, the parameters used for generating the small population. **Claim 8** A generation unit that generates a second small population included in a second generation, which is a generation after the first generation, based on first parameters that are parameters of an evolutionary algorithm set using the evaluation of a first small population included in the first generation. An evaluation unit that evaluates the generated second small population. A selection unit that selects second parameters, which are parameters of the evolutionary algorithm used for generating a third small population included in the second generation, based on the learning results using the evaluation of the second small population. An apparatus characterized by the above. **Claim 9** A step of generating a second small population included in a second generation, which is a generation after the first generation, based on first parameters that are parameters of an evolutionary algorithm set using the evaluation of a first small population included in the first generation. A step of evaluating the generated second small population. A step of selecting second parameters, which are parameters of the evolutionary algorithm used for generating a third small population included in the second generation, based on the learning results using the evaluation of the second small population. A method characterized by the above. **Claim 10** A computer is provided with: A process of generating a second small population included in a second generation, which is a generation after the first generation, based on first parameters that are parameters of an evolutionary algorithm set using the evaluation of a first small population included in the first generation. A process of evaluating the generated second small population. A program for causing the computer to execute a process of selecting second parameters, which are parameters of the evolutionary algorithm used for generating a third small population included in the second generation, based on the learning results using the evaluation of the second small population.

Images