Genetic programming symbol regression method based on multi-population asynchronous parallelism

Abstract

The invention discloses a genetic programming symbol regression method based on multi-population asynchronization and parallelism, which comprises the following steps of: constructing a sub-population of a symbol model based on a genetic algorithm, and taking the symbol model as an individual of the sub-population; applying for a shared memory for an elite group of the symbol model, and applying for a message queue used for collecting elite individuals in a sub-group of each symbol model; iteratively executing selection operation, crossover operation, mutation operation and fitness evaluation on each sub-population to optimize the sub-population; an independent thread is applied for executing a genetic programming algorithm for each sub-population, and a symbol model set is searched for the training data set of each sub-population; when each sub-population meets a sending condition, the sub-population sends an elite individual set in the population at the moment to a message queue; and after the termination condition of the genetic programming algorithm is met, performing primary sorting by adopting the root-mean-square error indexes of the elite individuals in the elite group, and taking the elite individual with the minimum root-mean-square error as a final output symbol model.

Description

Technical Field

[0001] The present invention belongs to the field of computational intelligence algorithms, and mainly relates to a genetic programming symbolic regression method based on asynchronous parallelism of multiple populations, which realizes asynchronous parallel effects by sharing elite symbolic models of multiple populations. Background Art

[0002] In actual symbolic regression problems, data features are usually high-dimensional and the amount of data is huge. The data features of regression problems in the symbolic regression benchmark library SRBench range from 2 to 124, and the number of instances ranges from 47 to 1,000,000, so the computational efficiency of the solution method is relatively high. The research on symbolic regression based on swarm intelligence optimization algorithms is relatively mature and effective, including genetic algorithms, particle swarm algorithms, artificial immune algorithms, ant colony algorithms, etc. They are all inspired by nature and imitate the biological behavior in the population to perform individual iterative optimization to obtain solutions that meet the constraints. They are widely used in academia and industry, among which genetic programming is the most widely used.

[0003] However, the computational efficiency of swarm intelligence optimization algorithms is often criticized. In order to improve the population diversity and search efficiency of the algorithm, scholars have proposed the idea of ​​multi-population co-evolution, which uses multi-core computers to evolve multiple sub-populations in parallel. The sub-populations interact to keep information synchronized, reducing the computational time exponentially while maintaining the total number of individual optimizations. Multi-population parallel algorithms are called coarse-grained parallel methods. The population interaction method is usually to migrate training data or individuals under a preset number of generations. Therefore, multiple sub-populations need to be synchronized under preset evolutionary conditions, resulting in mutual waiting when the sub-populations evolve, and information interaction is limited by the number and scale of interactions, resulting in overfitting of symbolic models. Summary of the invention

[0004] The present invention constructs a genetic programming symbolic regression method based on asynchronous parallelism of multiple populations by maintaining a shared symbolic model elite group that is updated in real time, so as to improve the efficiency of data calculation and search of the algorithm.

[0005] In order to achieve the above tasks, the present invention adopts the following technical solutions:

[0006] A genetic programming symbolic regression method based on multi-population asynchronous parallelism, comprising:

[0007] Construct subpopulations of symbolic models based on genetic algorithms, use binary trees to encode symbolic models, and use symbolic models as individuals of subpopulations; apply for shared memory for elite groups of symbolic models in multi-core computer devices for interaction between subpopulations; apply for message queues for collecting elite individuals in subpopulations of each symbolic model;

[0008] The input data of the genetic programming algorithm are sub-populations, training data sets of sub-populations, symbolic model sets, elite groups and message queues; for each sub-population, selection operations, crossover operations, mutation operations and fitness evaluation are iteratively performed to optimize the sub-population; an independent thread is applied for executing the genetic programming algorithm for each sub-population, and the symbolic model set is searched for the training data sets of each sub-population respectively;

[0009] When each sub-population meets the sending conditions, the sub-population sends the elite individuals in the population to the message queue;

[0010] When the termination condition of the genetic programming algorithm is met, the root mean square error index of the elite individuals in the elite group is used for sorting, and the elite individual with the smallest root mean square error is the final output symbolic model.

[0011] Furthermore, the elite group is set to be read-only for the sub-population, and the sub-population does not need to perform mutual exclusion operations when reading from the elite group or writing to the message queue.

[0012] Furthermore, under the condition of adopting the same symbolic model encoding method, different sub-populations can adopt different types of symbolic regression algorithms.

[0013] Furthermore, during the execution of the genetic programming algorithm, a thread for automatically maintaining the elite group is applied, the elite individual set in the message queue is continuously read, individuals in the elite group are updated by a tournament selection method, fitness values ​​are compared with random individuals in the elite group, and individuals with higher fitness values ​​are retained in the elite group.

[0014] Furthermore, the objective function of the best individual in the subpopulation is expressed as:

[0015]

[0016] Where F is a symbolic model set consisting of operators, data features and constants, and f is a symbolic model in the set, which is used to describe the data X in the training data set. i And the corresponding result y i The relationship between f(X i ) represents data X i After the output of the symbolic model f, X i is the i-th data in the data set, y i is the result corresponding to the data, represents the training data set, and N is the amount of data in the training data set.

[0017] Furthermore, different subsets of DS are selected by random sampling, etc. As the training data set of the i-th subpopulation.

[0018] Furthermore, the selection operation adopts tournament selection, the crossover operation adopts subtree crossover, etc., and the mutation operation adopts point compilation, subtree mutation and boosting mutation, etc.

[0019] Furthermore, before the sub-population performs the selection operation, the redundant individuals whose repetition times exceed the preset number of repetitions are eliminated. When the number of individuals in the sub-population is less than the preset value, it is considered that the evolutionary pressure is small, and a certain number of individuals are copied from the elite group to make up for it.

[0020] Furthermore, the sending condition of the sub-population is set as:

[0021] The elite individuals are sent to the message queue once at a preset interval. When the subpopulation adopts a multi-objective optimization algorithm, the Pareto frontier individuals are sent as elite individuals. When a single-objective optimization algorithm is adopted, a specified fixed number of current optimal individuals are sent as elite individuals.

[0022] Furthermore, when the RMSE of elite individuals is set to < 10 -16 Or when the number of generations is greater than or equal to the preset number of iterations MaxGen, it is the condition for terminating the algorithm.

[0023] A terminal device comprises a processor, a memory and a computer program stored in the memory; when the processor executes the computer program, the genetic programming symbolic regression method based on multi-population asynchronous parallelism is implemented.

[0024] Compared with the prior art, the present invention has the following technical features:

[0025] 1. The multi-population asynchronous evolution process of the present invention is relatively independent, and there will be no waiting problem in the multi-population evolution process. The interaction between sub-populations reuses the elite individual set information in the message queue to control the termination of the algorithm, saving memory space and improving the computing efficiency of the overall system.

[0026] 2. In the present invention, the frequency and quantity of information interaction among multiple sub-populations are controllable, and the shared elite group improves the search efficiency, increases the diversity of sub-populations, and effectively improves the search efficiency of the algorithm system.

[0027] 3. The method framework proposed in the present invention has strong versatility, and the elite retention method, update strategy, etc. can be replaced. It is suitable for different types of swarm intelligence optimization algorithms, and the application of the method is not limited to the field of symbolic regression. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a principle framework diagram of the method of the present invention;

[0029] Figure 2 It is a flow chart of the genetic programming algorithm combined with a shared elite population. DETAILED DESCRIPTION

[0030] The present invention designs a method for interaction between multiple populations to achieve the effect of asynchronous parallelism of multiple populations, while ensuring that information interaction will not affect the relatively independent evolutionary processes of multiple sub-populations, which can effectively improve population diversity and search efficiency. The present invention introduces a shared symbolic model elite group for indirect information interaction between multiple sub-populations. The elite group is updated in real time from the elite individual message queue sent by the sub-population. When the evolutionary pressure of the sub-population is too small, part of it is copied from the elite group as the offspring individual. The method mainly includes a symbolic model elite group and multiple relatively independent evolutionary sub-populations. The technical solution is as follows:

[0031] See attached Figure 1 and Figure 2 The present invention provides a genetic programming symbolic regression method based on multi-population asynchronous parallelism, comprising the following steps:

[0032] Step 1: construct a subpopulation of symbolic models based on a genetic algorithm, the number of subpopulations is n, the size of each subpopulation is set to PopSize, the maximum evolutionary generation is MaxGen, a binary tree is used to encode the symbolic model, and the symbolic model is used as an individual of the subpopulation;

[0033] Get the training dataset for the subpopulation Build the elite group of symbolic models and the message queue of elite individuals, including:

[0034] In a multi-core computer device, a shared memory is applied for the elite group of the symbolic model for interaction between the sub-populations; a message queue is applied for collecting the elite individuals in the sub-populations of each symbolic model; the elite group is set to be read-only for the sub-population; the elite individuals screened out from the sub-population enter the message queue, and the elite group is used to store the elite individuals.

[0035] Among them, when the sub-population reads from the elite group or writes to the message queue, there is no need to perform mutual exclusion operations, which will not cause other modules to block.

[0036] Step 2, the input data of the genetic programming algorithm are sub-populations, training data sets of sub-populations, symbolic model sets, elite groups and message queues; for each sub-population, selection operations, crossover operations, mutation operations, and fitness evaluation are iteratively performed to optimize the sub-population; an independent thread (process) is applied for executing the genetic programming algorithm for each sub-population, and the symbolic model set is searched for the training data set of each sub-population respectively; under the condition of using the same symbolic model encoding method, different sub-populations can use different types of symbolic regression algorithms.

[0037] During the execution of the genetic programming algorithm, apply for automatic maintenance of the elite group of threads (processes), such as Figure 1As shown in the dotted box, the elite individual set in the message queue is continuously read, the individuals in the elite group are updated by the tournament selection method, the fitness values ​​are compared with the random individuals in the elite group, the individuals with higher fitness values ​​are retained in the elite group, and whether to terminate this step is determined according to whether the set is an empty set.

[0038] In this step, the elite group does not need to be sorted in real time according to the fitness values ​​of its individuals (symbolic models), and can process the elite individual set in the message queue in a timely manner, which can ensure that the elite group preserves the high-quality individuals in the entire symbolic regression system, thereby achieving efficient interaction when multiple groups are asynchronously parallel.

[0039] In one embodiment of the present invention, to obtain a training data set describing a symbolic regression problem Symbolic model The objective function of the best individual in the subpopulation is expressed as:

[0040]

[0041] Where F is a symbolic model set consisting of operators, data features and constants, and f is a symbolic model in the set, which is used to describe the data X in the training data set. i And the corresponding result y i The relationship between f(X i ) represents data X i After the output of the symbolic model f, X i is the i-th data in the data set, y i The result corresponding to the data.

[0042] When the symbolic regression problem dataset When the scale is large, different subsets of DS can be selected by random sampling, etc. i sub As the training data set of the i-th subpopulation.

[0043] In this step, when comparing the fitness values ​​of elite individuals in the message queue and random individuals in the elite group, the same calculation method needs to be used, and the evaluated data sets are all the original data sets DS. If the sub-population does not use the original data set DS, the elite individuals in the message queue should be re-evaluated before comparison.

[0044] In this embodiment, the selection operation adopts tournament selection, the crossover operation adopts subtree crossover, etc., and the mutation operation adopts point compilation, subtree mutation, and boosting mutation, etc.

[0045] Before performing the selection operation in the sub-population, the redundant individuals whose repetition times exceed the preset value MaxSame are eliminated. When the number of individuals in the sub-population is less than the preset value PopSize, it is considered that the evolutionary pressure is small, and a certain number of individuals are copied from the elite group to make up for it.

[0046] Step 3, when each sub-population meets the sending condition, the sub-population sends the elite individual set in the population at this time to the message queue. During the evolution of the sub-population, the individuals of the population in adjacent generations are relatively similar. The sending condition of the present invention is set to send the elite individual set to the message queue once every Gap generation. When the sub-population adopts a multi-objective optimization algorithm, the Pareto frontier surface individuals are sent as elite individuals. When a single-objective optimization algorithm is adopted, a specified fixed number of current optimal individuals are sent as elite individuals.

[0047] Step 4: When the termination condition of the genetic programming algorithm is met, the RMSE index of the elite individuals in the elite group is used for sorting, and the elite individual with the smallest RMSE is the symbolic model of the final screening output.

[0048] Therefore, the number of evaluations of the symbolic model of the entire system is approximately n×MaxGen×PopSize, which is similar to the multi-population synchronous parallel method.

[0049] In one embodiment of the present invention, the root mean square error (RMSE) is used as the fitness evaluation indicator, and the elite individual's RMSE is set to be less than 10. -16 Or when the number of generations is greater than or equal to the preset number of iterations MaxGen, it is the condition for terminating the algorithm.

[0050] Example:

[0051] In order to test the computational efficiency and search efficiency of the method of the present invention, the 122 black box problems of the symbolic regression benchmark test set SRBench were compared with the multi-population synchronous parallel symbolic regression algorithm. Under the conditions of setting the number of subpopulations to 6, the maximum number of evolutionary generations to 300, and sending the elite individual set once every 10 generations, both algorithms were independently executed 10 times. The average execution time of the method of the present invention is improved by about 6% compared with the multi-population synchronous parallel method, and the test correlation coefficient R of the method of the present invention is 2.34. 2 The median is 0.81, and the median is 0.75 for the multi-group synchronous parallel method. The results show that the computational efficiency and search efficiency of the proposed method are improved compared with the multi-group synchronous parallel method.

[0052] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A genetic programming symbolic regression method based on multi-population asynchronous parallelism, characterized in that: include: A subpopulation of symbolic models is constructed based on a genetic algorithm, a binary tree is used to encode the symbolic models, and the symbolic models are used as individuals in the subpopulation; Apply for shared memory for the elite group of the symbolic model in the multi-core computer device for interaction between the sub-populations; apply for a message queue for collecting the elite individuals in the sub-populations of each symbolic model; The input data of the genetic programming algorithm are sub-populations, training data sets of sub-populations, symbolic model sets, elite groups and message queues; for each sub-population, selection operations, crossover operations, mutation operations and fitness evaluation are iteratively performed to optimize the sub-population; an independent thread is applied for executing the genetic programming algorithm for each sub-population, and the symbolic model set is searched for the training data sets of each sub-population respectively; When each sub-population meets the sending conditions, the sub-population sends the elite individuals in the population to the message queue; When the termination condition of the genetic programming algorithm is met, the root mean square error index of the elite individuals in the elite group is used for sorting, and the elite individual with the smallest root mean square error is the final output symbolic model.

2. The method for symbolic regression based on asynchronous parallelism of multiple populations according to claim 1, characterized in that: The elite group is set to be read-only for the sub-group, and the sub-group does not need to perform mutual exclusion operations when reading from the elite group or writing to the message queue.

3. The method of genetic programming symbolic regression based on multi-population asynchronous parallelism according to claim 1, characterized in that: Under the condition of using the same symbolic model encoding method, different sub-populations can adopt different types of symbolic regression algorithms.

4. The method for symbolic regression based on asynchronous parallelism of multiple populations according to claim 1, characterized in that: During the execution of the genetic programming algorithm, a thread for automatically maintaining the elite group is applied, the elite individual set in the message queue is continuously read, individuals in the elite group are updated by a tournament selection method, fitness values ​​are compared with random individuals in the elite group, and individuals with higher fitness values ​​are retained in the elite group.

5. The method of genetic programming symbolic regression based on multi-population asynchronous parallelism according to claim 1, characterized in that: The objective function of the best individual in the subpopulation is expressed as: Where F is a symbolic model set consisting of operators, data features and constants, and f is a symbolic model in the set, which is used to describe the data X in the training data set. i And the corresponding result y i The relationship between f(X i ) represents data X i After the output of the symbolic model f, X i is the i-th data in the data set, y i is the result corresponding to the data, represents the training data set, and N is the amount of data in the training data set.

6. The method of genetic programming symbolic regression based on multi-population asynchronous parallelism according to claim 1, characterized in that: Select different subsets of DS by random sampling, etc. As the training data set of the i-th subpopulation.

7. The method of genetic programming symbolic regression based on multi-population asynchronous parallelism according to claim 1, characterized in that: The selection operation adopts tournament selection, the crossover operation adopts subtree crossover, etc., and the mutation operation adopts point compilation, subtree mutation and boosting mutation, etc.

8. The method of genetic programming symbolic regression based on multi-population asynchronous parallelism according to claim 1, characterized in that: Before the selection operation is performed on the sub-population, redundant individuals whose repetition times exceed the preset number of repetitions are eliminated. When the number of individuals in the sub-population is less than the preset value, it is considered that the evolutionary pressure is small, and a certain number of individuals are copied from the elite group to make up for it.

9. The method of genetic programming symbolic regression based on multi-population asynchronous parallelism according to claim 1, characterized in that: The sending conditions of the sub-population are set as: The elite individuals are sent to the message queue once at a preset interval. When the subpopulation adopts a multi-objective optimization algorithm, the Pareto frontier individuals are sent as elite individuals. When a single-objective optimization algorithm is adopted, a specified fixed number of current optimal individuals are sent as elite individuals.

10. A terminal device comprising a processor, a memory and a computer program stored in the memory; characterized in that: When the processor executes the computer program, the method for genetic programming symbolic regression based on asynchronous parallelism of multiple populations according to any one of claims 1 to 9 is implemented.

Images