Automatic machine learning method and system based on multi-objective evolutionary optimization

Abstract

The invention relates to an automatic machine learning method and system based on multi-objective evolutionary optimization, and the method comprises the steps: screening out a plurality of purposes of hyper-parameters according to actual scene demands, obtaining initial hyper-parameter data through global search, carrying out the grouping according to the screened-out purposes of the hyper-parameters, carrying out the grouping evolutionary screening, and obtaining the initial hyper-parameter data; by means of the mode, historical data can be efficiently utilized, meanwhile, the method can be better close to an actual scene, the automation degree is high, the rationality of hyper-parameters can be better improved, and then the rationality of automatic machine learning in the later period is guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of privacy computing technology, and more specifically, to an automatic machine learning method and system based on multi-objective evolutionary optimization. Background Art

[0002] With the widespread application of machine learning in the field of privacy computing, traditional machine learning methods often require a lot of domain knowledge and tedious parameter adjustments when applied, which makes the model building process complicated and time-consuming. Therefore, automatic machine learning technology was born, which greatly reduces the difficulty of machine learning by automatically selecting models, adjusting hyperparameters and preprocessing data.

[0003] In automatic machine learning, the setting of hyperparameters is usually done before starting the learning process, rather than parameter data obtained through training. Usually, it is necessary to optimize the hyperparameters and select a set of optimal hyperparameters for the model to improve the performance and effect of learning. When faced with multiple conflicting optimization goals, optimizing one of the goals will have an adverse effect on other goals. Existing optimization methods are difficult to balance the relationship between the various goals. An automatic machine learning method and system based on multi-objective evolutionary optimization is needed. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide an automatic machine learning method based on multi-objective evolutionary optimization and an automatic machine learning system based on multi-objective evolutionary optimization in response to the above-mentioned defects of the prior art.

[0005] The technical solution adopted by the present invention to solve its technical problem is:

[0006] Construct an automatic machine learning method based on multi-objective evolutionary optimization, which includes the following steps:

[0007] Step 1: Construct a hyperparameter optimization task with multiple different purposes, and globally search historical hyperparameter data to establish an initial population of hyperparameters;

[0008] Step 2: Based on the set fitness, screen and obtain multiple advantageous hyperparameter sets corresponding to different purposes of the hyperparameter optimization task objectives;

[0009] Step 3: Select the optimal solution from each set of dominant hyperparameters and use it as the parent hyperparameter set;

[0010] Step 4: Cross the parent hyperparameter set with the data in each advantage hyperparameter set to obtain multiple new advantage hyperparameter sets;

[0011] Step 5: Evaluate whether multiple new advantageous hyperparameter sets have a global optimal solution that meets the requirements. If yes, proceed to the next step. Otherwise, return to step 3 and replace the original multiple advantageous hyperparameter sets with multiple new advantageous hyperparameter sets.

[0012] Step 6: Output the global optimal solution.

[0013] The automatic machine learning method based on multi-objective evolutionary optimization of the present invention, wherein the hyperparameter optimization task objectives with multiple different purposes are constructed using:

[0014] According to the category to which the hyperparameter belongs, obtain the hyperparameter function purpose data corresponding to the category in the historical data;

[0015] The hyperparameter function purpose data is screened to obtain multiple different purposes that meet the actual application scenarios.

[0016] The automatic machine learning method based on multi-objective evolutionary optimization of the present invention, wherein the hyperparameter data of the global search history is used to establish the initial population of hyperparameters:

[0017] Perform a global search for hyperparameter data of the same type and determine whether the number of obtained data exceeds the set value;

[0018] If it exceeds, the obtained data will be directly used as the initial population of hyperparameters;

[0019] If it does not exceed the limit, the obtained data will be mutated so that the number of data exceeds the set value, and the set of the obtained data and the mutated data will be used as the initial population of the hyperparameter.

[0020] The automatic machine learning method based on multi-objective evolutionary optimization of the present invention, wherein the step five also includes a method:

[0021] Count the number of operations to return to step 3;

[0022] When the count exceeds the set threshold, a mutation operation is performed on the new set of advantageous hyperparameters.

[0023] The automatic machine learning method based on multi-objective evolutionary optimization of the present invention, wherein the evaluation of whether multiple new sets of advantageous hyperparameters have a global optimal solution that meets the requirements adopts:

[0024] Determine whether there is a hyperparameter that can simultaneously meet the setting requirements of multiple different purposes of the hyperparameter optimization task objectives, and if so, it is considered to be the global optimal solution.

[0025] The automatic machine learning method based on multi-objective evolutionary optimization of the present invention, wherein the step five also includes a method:

[0026] Count the number of mutation operations on the new set of advantageous hyperparameters;

[0027] When the count exceeds a set threshold, the set requirements for multiple different purposes of the hyperparameter optimization task objectives are reduced according to the set rules.

[0028] An automatic machine learning system based on multi-objective evolutionary optimization, wherein the system comprises a task objective generation unit, a multi-objective evolutionary optimization unit and a data output unit;

[0029] The task target generation unit is used to construct a hyperparameter optimization task target with multiple different purposes;

[0030] The multi-objective evolutionary optimization unit is used to globally search historical hyperparameter data and establish an initial hyperparameter population; based on the set fitness, screen and obtain multiple advantageous hyperparameter sets corresponding to different purposes of the hyperparameter optimization task objectives; screen the optimal solution in each advantageous hyperparameter set and use it as the parent hyperparameter set; cross the parent hyperparameter set with the data in each advantageous hyperparameter set to obtain multiple new advantageous hyperparameter sets; evaluate whether the multiple new advantageous hyperparameter sets have a global optimal solution that meets the requirements, and if so, the data output unit outputs the global optimal solution, otherwise, return to the step of screening the optimal solution in each advantageous hyperparameter set, and at this time replace the original multiple advantageous hyperparameter sets with the multiple new advantageous hyperparameter sets;

[0031] The data output unit is used to output the obtained global optimal solution.

[0032] The automatic machine learning system based on multi-objective evolutionary optimization of the present invention, wherein the hyperparameter optimization task objectives with multiple different purposes are constructed using:

[0033] According to the category to which the hyperparameter belongs, obtain the hyperparameter function purpose data corresponding to the category in the historical data;

[0034] The hyperparameter function purpose data is screened to obtain multiple different purposes that meet the actual application scenarios.

[0035] The automatic machine learning system based on multi-objective evolutionary optimization of the present invention, wherein the hyperparameter data of the global search history is used to establish the initial population of hyperparameters:

[0036] Perform a global search for hyperparameter data of the same type and determine whether the number of obtained data exceeds the set value;

[0037] If it exceeds, the obtained data will be directly used as the initial population of hyperparameters;

[0038] If it does not exceed the limit, the obtained data will be mutated so that the number of data exceeds the set value, and the set of the obtained data and the mutated data will be used as the initial population of the hyperparameter.

[0039] The automatic machine learning system based on multi-objective evolutionary optimization of the present invention, wherein the multi-objective evolutionary optimization unit is further used for:

[0040] Count the number of operations of returning the optimal solution step in each advantageous hyperparameter set;

[0041] When the count exceeds the set threshold, a mutation operation is performed on the new set of advantageous hyperparameters;

[0042] The method of evaluating whether multiple new advantageous hyperparameter sets have a global optimal solution that meets the requirements adopts:

[0043] Determine whether there is a hyperparameter that can simultaneously meet the setting requirements of multiple different purposes of the hyperparameter optimization task objectives, and if so, identify it as the global optimal solution;

[0044] The multi-objective evolutionary optimization unit is also used for:

[0045] Count the number of mutation operations on the new set of advantageous hyperparameters;

[0046] When the count exceeds a set threshold, the set requirements for multiple different purposes of the hyperparameter optimization task objectives are reduced according to the set rules.

[0047] The beneficial effect of the present invention is that by applying the method of the present application, when it is necessary to optimize hyperparameters, multiple purposes of the hyperparameters are screened out according to the actual scenario requirements, and initial hyperparameter data is obtained through global search, and grouping is performed according to the multiple purposes of the screened hyperparameters, and then group evolution screening is performed until the optimal solution that meets the requirements of multiple purposes is generated through iteration. In this way, historical data can be efficiently utilized while being better adapted to actual scenarios, with a high degree of automation and better ability to improve the rationality of hyperparameters, thereby ensuring the rationality of subsequent automatic machine learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the present invention will be further described below in conjunction with the accompanying drawings and embodiments. The drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work:

[0049] Figure 1 It is a flow chart of an automatic machine learning method based on multi-objective evolutionary optimization according to a preferred embodiment of the present invention;

[0050] Figure 2 It is a principle block diagram of an automatic machine learning system based on multi-objective evolutionary optimization in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the following will be described clearly and completely in combination with the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are partial embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the protection scope of the present invention.

[0052] The automatic machine learning method based on multi-objective evolutionary optimization in the preferred embodiment of the present invention is as follows: Figure 1 As shown, the following steps are included:

[0053] S01: Construct a hyperparameter optimization task with multiple different purposes, and globally search historical hyperparameter data to establish an initial population of hyperparameters;

[0054] During implementation, the hyperparameter function purpose data corresponding to the category in the historical data can be obtained according to the category to which the hyperparameter belongs; the hyperparameter function purpose data can be screened to obtain multiple different purposes that meet the actual application scenarios;

[0055] Perform a global search for hyperparameter data of the same type and determine whether the number of data obtained exceeds the set value; if so, use the obtained data directly as the initial population of the hyperparameter; if not, perform a mutation operation on the obtained data so that the number of data exceeds the set value, and use the set of the obtained data and the mutated data as the initial population of the hyperparameter;

[0056] In this way, it is possible to ensure that the initial data has sufficient characteristics and quantity;

[0057] S02: Based on the set fitness, screen and obtain multiple advantageous hyperparameter sets corresponding to different purposes of the hyperparameter optimization task objectives;

[0058] This step is different from the conventional operation mode. The difference is that the purpose-oriented screening of advantageous parameters can better include parameter value data that meets the requirements and provide materials for subsequent cross evolution. This step can also greatly improve the probability of obtaining the global optimal solution.

[0059] S03: Filter the optimal solution among each dominant hyperparameter set and use it as the parent hyperparameter set;

[0060] The optimal solution described in this step, that is, the optimal parameters for solving a project, is also the extreme value of each set of advantageous hyperparameters;

[0061] S04: Cross the parent hyperparameter set with the data in each advantage hyperparameter set to obtain multiple new advantage hyperparameter sets;

[0062] The existing method can be adopted for crossover, and no further description is given;

[0063] S05: Evaluate whether multiple new advantage hyperparameter sets have a global optimal solution that meets the requirements. If yes, proceed to the next step. Otherwise, return to step 3 and replace the original multiple advantage hyperparameter sets with the multiple new advantage hyperparameter sets.

[0064] This step adds a judgment operation. In practical applications, this cycle usually involves multiple times. In order to ensure the specificity of the parameters, the number of operations returning to step three can be counted; when the count exceeds the set threshold, the new set of advantageous hyperparameters is mutated;

[0065] Furthermore, the global optimal solution is determined by: judging whether there is a hyperparameter that can simultaneously meet the setting requirements of multiple different purposes of the hyperparameter optimization task objectives, and if so, it is determined to be the global optimal solution.

[0066] In order to avoid setting the requirements of the purpose too high, you can use the following method: count the number of mutation operations on the new advantage hyperparameter set; when the count exceeds the set threshold, reduce the setting requirements of multiple different purposes of the hyperparameter optimization task goals according to the set rules; of course, the reduction of requirements here is not to reduce the overall performance, but only to prevent the situation where the requirements are set too high due to personal operation;

[0067] S06: Output the global optimal solution;

[0068] By applying the method of the present application, when it is necessary to optimize hyperparameters, multiple purposes of hyperparameters are screened out according to the actual scenario requirements, and initial hyperparameter data is obtained through global search, and grouped according to the multiple purposes of the screened hyperparameters, and then group evolution screening is performed until the optimal solution that meets the requirements of multiple purposes is generated through iteration. This method can efficiently utilize historical data while being better adapted to actual scenarios, with a high degree of automation and better ability to improve the rationality of hyperparameters, thereby ensuring the rationality of subsequent automatic machine learning.

[0069] An automatic machine learning system based on multi-objective evolutionary optimization, such as Figure 2 As shown, the system includes a task target generation unit 100, a multi-target evolutionary optimization unit 101 and a data output unit 102;

[0070] A task target generation unit 100 is used to construct a hyperparameter optimization task target with multiple different purposes;

[0071] The multi-objective evolutionary optimization unit 101 is used for globally searching historical hyperparameter data and establishing an initial hyperparameter population; based on the set fitness, screening and obtaining multiple advantageous hyperparameter sets corresponding to different purposes of the hyperparameter optimization task objectives; screening the optimal solution in each advantageous hyperparameter set and using it as the parent hyperparameter set; crossing the parent hyperparameter set with the data in each advantageous hyperparameter set to obtain multiple new advantageous hyperparameter sets; evaluating whether the multiple new advantageous hyperparameter sets have a global optimal solution that meets the requirements, and if so, the data output unit outputs the global optimal solution, otherwise, returning to the step of screening the optimal solution in each advantageous hyperparameter set, and at this time replacing the original multiple advantageous hyperparameter sets with multiple new advantageous hyperparameter sets;

[0072] A data output unit 102, used to output the obtained global optimal solution;

[0073] By applying the system of the present application, when it is necessary to optimize hyperparameters, multiple purposes of hyperparameters are screened out according to actual scenario requirements, and initial hyperparameter data is obtained through global search. The screened hyperparameters are grouped according to the multiple purposes, and then group evolution screening is performed until the optimal solution that meets the requirements of multiple purposes is generated through iteration. This method can efficiently utilize historical data while being better adapted to actual scenarios. It has a high degree of automation and can better improve the rationality of hyperparameters, thereby ensuring the rationality of subsequent automatic machine learning.

[0074] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all these improvements and changes should fall within the scope of protection of the appended claims of the present invention.

Claims

1. An automatic machine learning method based on multi-objective evolutionary optimization, characterized in that: The following steps are involved: Step 1: Construct a hyperparameter optimization task with multiple different purposes, and globally search historical hyperparameter data to establish an initial population of hyperparameters; Step 2: Based on the set fitness, screen and obtain multiple advantageous hyperparameter sets corresponding to different purposes of the hyperparameter optimization task objectives; Step 3: Select the optimal solution from each set of dominant hyperparameters and use it as the parent hyperparameter set; Step 4: Cross the parent hyperparameter set with the data in each advantage hyperparameter set to obtain multiple new advantage hyperparameter sets; Step 5: Evaluate whether multiple new advantageous hyperparameter sets have a global optimal solution that meets the requirements. If yes, proceed to the next step. Otherwise, return to step 3 and replace the original multiple advantageous hyperparameter sets with multiple new advantageous hyperparameter sets. Step 6: Output the global optimal solution.

2. The automatic machine learning method based on multi-objective evolutionary optimization according to claim 1, characterized in that: The goal of constructing a hyperparameter optimization task with multiple different purposes is to use: According to the category to which the hyperparameter belongs, obtain the hyperparameter function purpose data corresponding to the category in the historical data; The hyperparameter function purpose data is screened to obtain multiple different purposes that meet the actual application scenarios.

3. The automatic machine learning method based on multi-objective evolutionary optimization according to claim 1 is characterized in that: The global search history hyperparameter data is used to establish the initial population of hyperparameters: Perform a global search for hyperparameter data of the same type and determine whether the number of obtained data exceeds the set value; If it exceeds, the obtained data will be directly used as the initial population of hyperparameters; If it does not exceed the limit, the obtained data will be mutated so that the number of data exceeds the set value, and the set of the obtained data and the mutated data will be used as the initial population of the hyperparameter.

4. The automatic machine learning method based on multi-objective evolutionary optimization according to claim 1, characterized in that: The step five also includes a method: Count the number of operations to return to step 3; When the count exceeds the set threshold, a mutation operation is performed on the new set of advantageous hyperparameters.

5. The automatic machine learning method based on multi-objective evolutionary optimization according to claim 4 is characterized in that: The method of evaluating whether multiple new advantageous hyperparameter sets have a global optimal solution that meets the requirements adopts: Determine whether there is a hyperparameter that can simultaneously meet the setting requirements of multiple different purposes of the hyperparameter optimization task objectives, and if so, it is considered to be the global optimal solution.

6. The automatic machine learning method based on multi-objective evolutionary optimization according to claim 5, characterized in that: The step five also includes a method: Count the number of mutation operations on the new set of advantageous hyperparameters; When the count exceeds a set threshold, the set requirements for multiple different purposes of the hyperparameter optimization task objectives are reduced according to the set rules.

7. An automatic machine learning system based on multi-objective evolutionary optimization, characterized in that: The system includes a task target generation unit, a multi-target evolutionary optimization unit and a data output unit; The task target generation unit is used to construct a hyperparameter optimization task target with multiple different purposes; The multi-objective evolutionary optimization unit is used to globally search historical hyperparameter data and establish an initial hyperparameter population; based on the set fitness, screen and obtain multiple advantageous hyperparameter sets corresponding to different purposes of the hyperparameter optimization task objectives; screen the optimal solution in each advantageous hyperparameter set and use it as the parent hyperparameter set; cross the parent hyperparameter set with the data in each advantageous hyperparameter set to obtain multiple new advantageous hyperparameter sets; evaluate whether the multiple new advantageous hyperparameter sets have a global optimal solution that meets the requirements, and if so, the data output unit outputs the global optimal solution, otherwise, return to the step of screening the optimal solution in each advantageous hyperparameter set, and at this time replace the original multiple advantageous hyperparameter sets with the multiple new advantageous hyperparameter sets; The data output unit is used to output the obtained global optimal solution.

8. The automatic machine learning system based on multi-objective evolutionary optimization according to claim 7, characterized in that: The goal of constructing a hyperparameter optimization task with multiple different purposes is to use: According to the category to which the hyperparameter belongs, obtain the hyperparameter function purpose data corresponding to the category in the historical data; The hyperparameter function purpose data is screened to obtain multiple different purposes that meet the actual application scenarios.

9. The automatic machine learning system based on multi-objective evolutionary optimization according to claim 7, characterized in that: The global search history hyperparameter data is used to establish the initial population of hyperparameters: Perform a global search for hyperparameter data of the same type and determine whether the number of obtained data exceeds the set value; If it exceeds, the obtained data will be directly used as the initial population of hyperparameters; If it does not exceed the limit, the obtained data will be mutated so that the number of data exceeds the set value, and the set of the obtained data and the mutated data will be used as the initial population of the hyperparameter.

10. The automatic machine learning system based on multi-objective evolutionary optimization according to claim 7, characterized in that: The multi-objective evolutionary optimization unit is also used for: Count the number of operations of returning the optimal solution step in each advantageous hyperparameter set; When the count exceeds the set threshold, a mutation operation is performed on the new set of advantageous hyperparameters; The method of evaluating whether multiple new advantageous hyperparameter sets have a global optimal solution that meets the requirements adopts: Determine whether there is a hyperparameter that can simultaneously meet the setting requirements of multiple different purposes of the hyperparameter optimization task objectives, and if so, identify it as the global optimal solution; The multi-objective evolutionary optimization unit is also used for: Count the number of mutation operations on the new set of advantageous hyperparameters; When the count exceeds a set threshold, the set requirements for multiple different purposes of the hyperparameter optimization task objectives are reduced according to the set rules.

Images