Base model pool generation method based on multi-objective evolutionary optimization
By generating a base model pool through multi-objective evolutionary optimization, the problems of long time consumption and low data utilization in parallel ensemble learning are solved, and the generalization performance of the ensemble model is improved.
Patent Information
- Application Number
- CN202210208397.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-04
- Publication Date
- 2025-09-30
- Estimated Expiration
- 2042-03-04
AI Technical Summary
In existing parallel ensemble learning, base model generation is time-consuming and has low data utilization, making it difficult to ensure the generalization performance of the ensemble model.
A multi-objective evolutionary optimization method is used to generate a base model pool. Through sample data preprocessing, partitioning, base model structure selection, multi-objective evolutionary optimization training and optimization, base models that are not mutually dominant are screened out to build a high-quality integrated model.
It improves data utilization and base model quality, enhances the generalization ability of the integrated model, and reduces generation time.
Smart Images
Figure CN114692896B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of machine learning, and in particular relates to a method for intelligently generating a base model pool for ensemble learning. Background Art
[0002] A single learner is likely to be either underfitting or overfitting. In order to obtain a learner with excellent generalization performance, multiple individual learners can be trained and, through a certain fusion strategy, eventually form a strong learner, or an ensemble learning model.
[0003] Individual learners are also called base models. Based on the properties of the base models, ensemble models can be divided into two categories: homogeneous models and heterogeneous models. Homogeneous models are ensemble models constructed from the same type of base models, generated by the same algorithm and differing only in parameters; heterogeneous models are ensemble models constructed from different types of base models. Currently, homogeneous models are widely used, and the ensemble learning methods we often refer to are generally based on homogeneous models. Homogeneous models can be divided into two categories based on the dependencies between base models: serial and parallel. The serial approach leverages the dependencies between base models to improve model performance by assigning a larger weight to misclassified samples, a typical example being the Boosting algorithm. The parallel approach leverages the independence of base models to significantly reduce error through averaging, a typical example being the Bagging algorithm.
[0004] The basic principle of bagging to obtain a base model is to obtain multiple sets of non-identical training sets through random sampling (bootstrap) or data partitioning. A base model is then trained based on each training set, and a single base model can be obtained from each training set. The principle of this method for obtaining a base model can be summarized as follows: multiple training sets are obtained through multiple data partitioning, multiple base models are trained, and then an ensemble model is constructed. In parallel ensemble bagging, the most common method for generating base models currently uses this method. However, this base model generation method is time-consuming, has low data utilization per set, and lacks a good quality metric for each generated base model, making it difficult to guarantee the quality of the base models obtained during each training session, and even more difficult to guarantee the generalization ability of the ensemble model. Summary of the Invention
[0005] The purpose of the present invention is to address the problems of long base model generation time and low data utilization in current parallel ensemble models, and to provide a base model pool generation method based on multi-objective evolutionary optimization to enrich the existing base model generation methods in ensemble learning and improve data utilization. At the same time, performance evaluation indicators of the base model are also given to provide a higher quality base model for ensemble learning, so that the constructed ensemble model has stronger generalization ability.
[0006] To achieve the above object, the present invention adopts the following technical solutions:
[0007] The base model pool generation method based on multi-objective evolutionary optimization includes the following steps:
[0008] Step 1: Sample data preprocessing: Obtain sample data and preprocess the sample data into the pattern required by the algorithm;
[0009] Step 2: Sample data division: Divide the preprocessed sample data into training set T, validation set V and test set M;
[0010] Step 3: Base model structure selection: Determine the type and hyperparameters of the base model required for ensemble learning based on the ensemble model to be built;
[0011] Step 4: Base model pool training: Use a multi-objective evolutionary optimization method to learn the training set T and obtain a set of non-dominant data as the base model S for selection;
[0012] Step 5: Base model selection: Use the validation set V as the input of the selected base model S obtained in step 4, obtain the output set corresponding to the validation set V, and use this output set and the true output of the validation set V to calculate the mean square error (MSE) of the base model; then, combine the specified rules to select a specified number of base models from the selected base model S obtained in step 4;
[0013] Step 6: Use the base model obtained in step 5 as the base model for ensemble learning and construct the ensemble model according to the parallel ensemble method.
[0014] Furthermore, in step 4, the multi-objective evolutionary optimization method is used to train the base model hyperparameters to obtain a group of base models that are not mutually dominant. The detailed process is as follows:
[0015] Step 4.1. Initialize base models: Based on the base model types and hyperparameters determined in step 3, initialize a set of base models and determine the mapping rules from the base model hyperparameters to the solutions of the multi-objective optimization method.
[0016] Step 4.2, base model fitness evaluation: Input the training set T into the base model obtained in step 1, and calculate the fitness value of the base model according to the optimization goal of the preset multi-objective optimization problem;
[0017] Step 4.3: Base model evolution iteration: Optimize the hyperparameters of each base model through evolutionary operator iteration to promote the evolution of the base model.
[0018] Step 4.4, Base Model Pool Management: Using the base model hyperparameters obtained in step 4.3, solve the Pareto threshold frontier S of the required base model generation problem. If the preset multi-objective evolutionary optimization method stopping condition is reached, the experiment is stopped. The Pareto threshold frontier S at this time is the solution set of the selected optimal base model. Otherwise, continue with steps 4.2 to 4.4.
[0019] Furthermore, the above-mentioned method for generating a base model pool in ensemble learning further includes: step 7, evaluating the generalization performance of the ensemble model, wherein the specific evaluation method is:
[0020] Input the test set M into the integrated model obtained in step 6 and obtain an output set through calculation. Calculate the mean square error (MSE) between the output set obtained by the integrated model and the true output set corresponding to the test set M to evaluate the generalization performance of the integrated model. The smaller the mean square error, the stronger the generalization performance of the integrated model.
[0021] Furthermore, the sample data in step 1 may be real data in actual applications, or data collected in a test function (DTLZ, ZDT, WFG, etc.).
[0022] Furthermore, the preprocessing of the sample data in step 1 is achieved by filtering, denoising, whitening or normalization.
[0023] Furthermore, the multi-objective optimization method used in step 4 may be any one of the multi-objective evolutionary optimization algorithms (MOEA) such as the multi-objective genetic algorithm (MOGA) or the multi-objective particle swarm optimization algorithm.
[0024] Furthermore, the optimization objectives of the multi-objective optimization generation problem preset in step 4.2 include: minimizing the prediction error of each base model in the multi-objective optimization and minimizing the vector norm of the parameters constituting each base model; wherein the minimization of the prediction error of each base model in the multi-objective optimization represents the generalization performance target of the base model, and the minimization of the vector norm of the parameters constituting each base model represents the structural simplicity target; the prediction error includes but is not limited to minimizing the prediction mean square error MSE, minimizing the prediction mean absolute error MAE, and minimizing the prediction root mean square error RMSE; the vector norm includes but is not limited to minimizing the 1-norm ||w||1 and minimizing the 2-norm ||w||2.
[0025] Furthermore, when the mean square error MSE of the base model in step 5 is combined with specified rules to screen the base model, the specified rules adopted include but are not limited to: selecting several base models with smaller prediction errors (for example, the top 20%) and base models located at the inflection point on the Pareto threshold frontier surface. When using, any one of them can be selected.
[0026] Furthermore, when dividing the sample data in step 2, it is also possible to adopt a method of combining multiple divisions of the data to obtain multiple base models with a multi-objective optimization method. By combining the synergy of multiple divisions of the data to obtain multiple base models and the multi-objective optimization method, the generalization performance of the integrated model can be further improved.
[0027] The present invention provides a method for generating a base model pool based on multi-objective evolutionary optimization. The method first initializes several base models using a training set T and ensemble model information, and constructs mapping rules from the base models to the solutions of the multi-objective optimization method. A multi-objective optimization method is then used to solve the Pareto threshold frontier S of the base model generation problem in ensemble learning, where S represents a set of solutions for selecting base models. A specified number of base models are selected based on a validation set V and specified screening rules. Finally, the selected base models are used as base models for ensemble learning, and the ensemble model is constructed using a parallel ensemble method.
[0028] Compared with the prior art, the present invention has the following advantages:
[0029] 1. In the present invention, the Pareto threshold frontier obtained by the multi-objective optimization method is composed of a group of mutually non-dominant base models, and the selected base model is of higher quality.
[0030] 2. High data utilization: Compared with the traditional algorithm in which a group of data can only obtain one model after being divided once, the present invention can obtain multiple different models after a group of data is divided once.
[0031] 3. In the model pool of the present invention, each model has its own fitness. When used, different models can be selected according to actual needs. Therefore, a better base model can be selected scientifically and effectively to build an integrated model with stronger generalization ability. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 A comparison diagram of the traditional base model generation method and the base model generation method of the present invention;
[0033] Figure 2 Flowchart of the present invention;
[0034] Figure 3 This is a flowchart of the base model pool training in the embodiment. DETAILED DESCRIPTION
[0035] The present invention will be described in further detail below with reference to the accompanying drawings and embodiments.
[0036] One commonly used method for generating base models is to obtain base models through data partitioning training. This method generates base models and constructs an ensemble model by first obtaining multiple training sets through multiple data partitioning. Then, using these multiple training sets, a corresponding set of base models is obtained for each training set. Finally, these base models are used to construct the ensemble model. This base model generation method generates only one set of base models from one training set, resulting in low data utilization and weak quality control over the generated base models, making it difficult to guarantee the generalization performance of the ensemble model.
[0037] In order to enrich the generation method of the base model in the integrated model, provide a more quality-assured method for the generation of the base model, and improve the generalization ability of the integrated model, this embodiment provides a base model pool generation method based on multi-objective evolutionary optimization, such as Figure 2 As shown, the following steps are included:
[0038] Step 1: Sample data preprocessing: Sample data is obtained and preprocessed into the pattern required by the algorithm. In this embodiment, the raw data can be real data from actual applications or data collected by test functions (such as DTLZ, ZDT, and WFG). Sample data preprocessing can be achieved using methods such as filtering, denoising, whitening, or normalization.
[0039] Step 2: Sample Data Partitioning: The preprocessed sample data is divided into a training set T, a validation set V, and a test set M. Sample data partitioning methods disclosed in the prior art include holdout, cross-validation, and bootstrap methods, all of which can be used for sample data partitioning in this embodiment. During implementation, any of the sample data partitioning methods disclosed in the prior art can be used.
[0040] Step 3: Base model structure selection: Determine the structure of the base model required for ensemble learning based on the ensemble model to be built, that is, the type of base model (neural network model, decision tree model, etc.) and hyperparameters;
[0041] Step 4, base model pool training: Use the multi-objective evolutionary optimization method to learn the training set T, and solve the Pareto threshold frontier S of the base model generation multi-objective problem through training, that is, use the parameters of the base model as decision variables and build a multi-objective optimization model of the base model in the integrated model to search for a set of high-precision and differentiated selection base models S. In specific implementation, the base model generation multi-objective problem is optimized and solved in two aspects: minimizing the prediction error of the base model and minimizing the vector norm of the base model parameters. The detailed process is as follows Figure 3 As shown, proceed as follows:
[0042] Step 4.1. Initialize the base model: Based on the base model type and hyperparameters determined in step 3, initialize a set of base models and determine the mapping rules from the base model hyperparameters to the solutions of the multi-objective optimization method. The specific mapping rules are as follows: According to the model type and hyperparameters, determine the number of hyperparameters of the base model, then initialize the base model, and randomly generate multiple solutions of corresponding dimensions for multi-objective algorithm optimization. After optimization, for a solution found, according to the base model type and the number of hyperparameters, put the parameters of the found solution in each dimension back into the base model to complete the mapping.
[0043] Step 4.2, base model fitness evaluation: Input the training set T into the base model obtained in step 1, and calculate the fitness value of the base model according to the optimization goal of the preset multi-objective optimization problem;
[0044] Step 4.3: Base model evolution iteration: Optimize the hyperparameters of each base model through evolutionary operator iteration to promote the evolution of the base model.
[0045] Step 4.4, Base Model Pool Management: Use the base model hyperparameters obtained in Step 4.3 to maintain an external archive of the multi-objective optimization method. The solutions in the external archive consist of the Pareto threshold frontier S of the multi-objective base model generation problem. If the pre-set stopping condition of the multi-objective evolutionary optimization method is met, the experiment stops. Otherwise, return to Step 4.2 and continue until the condition is met.
[0046] Step 5, base model optimization: take the validation set V as the input of the base model, obtain the output set corresponding to the validation set V, and use the output set and the true output set of the validation set V to calculate the mean square error (MSE) of the base model; then, combine the specified rules to screen out a specified number of base models from the selected base models S obtained in step 4; in implementation, in order to obtain the optimal model, specify the base model with the smallest prediction error in the top 20%, the base model at the inflection point on the Pareto threshold frontier, etc. as a screening scheme combined with the mean square error.
[0047] Step 6: Use the base model obtained in step 5 as the base model for ensemble learning and construct the ensemble model according to the parallel ensemble method.
[0048] Step 7: Evaluate the generalization performance of the integrated model. The specific evaluation method is as follows:
[0049] Input the test set M into the integrated model obtained in step 6, and obtain an output set through calculation; use the mean square error (MSE) between the output set obtained by the integrated model using the test set M and the known true output set corresponding to the test set M to evaluate the generalization performance of the integrated model; the smaller the mean square error, the stronger the generalization performance of the integrated model.
[0050] Example 1
[0051] The embodiment of the present invention is further described by taking a randomly collected data set on a multi-objective test function DTLZ1 as basic data. In the embodiment of the present invention, the data set sizes are 900 and 300 respectively, and the data set dimension is 30.
[0052] Step 1: Normalize and preprocess the collected basic data, and then use the holdout method to divide the data into a ratio of 8:1:1 to obtain the training set T, validation set V and test set M.
[0053] Step 2: According to the integrated model to be constructed, the base model to be generated is determined to be a neural network model. The number of hidden layers of the neural network is 2, including 5 and 3 neurons respectively.
[0054] Step 3: Based on the model structure determined in step 2, construct several initialization base models and determine the mapping rules from the solution of the multi-objective optimization method to the base models.
[0055] The two objectives for solving the multi-objective problem of base model generation are to minimize the mean squared error (MSE) of the neural network predictions and to minimize the 2-norm ||w||2 of the neural network base model parameters. Based on the characteristics of the base model generation problem, a multi-objective particle swarm optimization algorithm is used to search for a set of selected base models using a training set T. Each solution set is represented according to a preset data structure, namely, as a solution vector. The dimension of the solution vector is determined by the input and output datasets and the base model structure. Each element of the solution vector identifies a parameter of a base model; in other words, a solution represents a base model.
[0056] Step 4: Select the base model with the smallest prediction mean square error (MSE) in the top 20% according to the base model screening rules, and construct it into an integrated model using the parallel integration method.
[0057] Step 5: Evaluate the generalization performance of the integrated model obtained in step 4. The specific evaluation method is as follows:
[0058] Input the test set M into the integrated model obtained in step 6, and obtain an output set through calculation; use the output set obtained by the integrated model of the test set M and the true output set corresponding to the known test set M to calculate the mean square error (MSE).
[0059] To verify the generalization performance of the base model pool generation method in ensemble learning in this embodiment, this embodiment performs the following operations:
[0060] First, we collected 30 sets of data using the multi-objective test function DTLZ1, each with a specified base data size. Each set of data was run independently 30 times, and the generalization performance of the ensemble model obtained by this method was calculated. Finally, we calculated the average of these 30 sets of generalization performance to represent the generalization performance of the ensemble model in statistical comparisons.
[0061] The ensemble model generated by the above method was then used as the DTLZ1 ensemble model, and the generalization performance was compared with that of the neural network ensemble model obtained by training the DTLZ1 using the BP algorithm. The comparison results showed that for a data set with a data size N of 900, the average prediction mean square error (MSE) of the former ensemble model was 2.640E-2, while that of the latter ensemble model was 5.452E-2. For a data set with a data size N of 300, the average prediction mean square error (MSE) of the former ensemble model was 2.799E-2, while that of the latter ensemble model was 5.766E-2.
[0062] Example 2
[0063] The collected data of 81 (Ce0.4Nd0.6)32.7FeB and Nd2Fe14B are used as the basic data to further describe the embodiment of the present invention. The collected data includes three model input data of temperature, deformation speed and deformation amount and coercivity H cj and remanence B r Both models output data.
[0064] Step 1: Normalize and preprocess the collected basic data, and then use the holdout method to divide the data into a ratio of 8:1:1 to obtain the training set T, validation set V and test set M.
[0065] Step 2: Determine the proxy model as an integrated neural network model as needed, determine the base model to be generated as a neural network model, and the number of hidden layers of the neural network is 2, including 5 and 3 neurons respectively.
[0066] Step 3: Based on the base model structure determined in step 2, several initialized base models are constructed, and the mapping rules from the solution of the multi-objective optimization method to the base model are determined.
[0067] The two objectives for solving the multi-objective problem of base model generation are to minimize the mean squared error (MSE) of the neural network predictions and to minimize the 2-norm ||w||2 of the neural network base model parameters. Based on the characteristics of the base model generation problem, a multi-objective particle swarm optimization algorithm is used to search for a set of selected base models using a training set T. Each solution set is represented according to a preset data structure, namely, as a solution vector. The dimension of the solution vector is determined by the input and output datasets and the base model structure. Each element of the solution vector identifies a parameter of a base model; in other words, a solution represents a base model.
[0068] Step 4: Select the base model with the smallest prediction mean square error (MSE) in the top 20% according to the base model screening rules, and construct it into an integrated model using the parallel integration method.
[0069] Step 5: Input the test set M into the integrated model obtained in step 4 and obtain an output set through calculation; use the output set obtained by the integrated model of the test set M and the true output set corresponding to the known test set M to calculate the mean square error (MSE) to characterize the generalization ability of the integrated model.
[0070] It can be seen from Examples 1 and 2 that the Pareto threshold frontier obtained by the base model pool generation method in the ensemble learning of the present invention using a multi-objective optimization method is composed of a group of non-dominant base models. During the whole process, a set of data can generate multiple different base models. Figure 1 The figure is a comparison between the traditional base model generation method and the base film generation method of the present invention; the upper part is the traditional method of obtaining multiple base models by multiple data divisions, and the lower part is the base model generation method of this embodiment. Figure 1 It can be clearly concluded that the utilization rate of data in the base film generation process of this embodiment is higher.
[0071] Based on the idea of the base model pool generation method in the above-mentioned ensemble learning, in the embodiments of the present invention, it is also possible to generate base models by combining it with the currently commonly used data partitioning method in which multiple groups of base models are obtained through multiple data partitioning. Specifically:
[0072] (1) Sample preprocessing and sample partitioning: Preprocess the sample data into the pattern required by the algorithm, and divide the data set according to the specified ratio to obtain the test set M. The remaining data is divided multiple times according to the specified ratio to obtain multiple sets of training sets T and validation sets V that are not completely the same.
[0073] (2) Base model structure selection: Confirm the base model structure to be generated in ensemble learning.
[0074] (3) Base model pool training and optimization. For each different set of training sets and validation sets, the base model pool generation method in the ensemble learning based on evolutionary multi-objective optimization is used to obtain multiple sets of base models, and the base models are selected according to the specified screening rules.
[0075] (4) Base model integration: All selected base models are merged according to the parallel integration method to construct an integrated model.
[0076] The synergy between the two further improves the generalization performance of the ensemble model. Taking a dataset with a data size of N = 900 as an example, the average prediction mean square error (MSE) of the ensemble generated by combining the above model with the idea of obtaining multiple base models through traditional data partitioning is 2.380E-2, a significant improvement over the original MSE of 2.640E-2. For a dataset with a data size of N = 300, the average prediction mean square error (MSE) of the ensemble generated by combining the above model with the idea of obtaining multiple base models through traditional data partitioning is 2.479E-2, a significant improvement over the original MSE of 2.799E-2.
[0077] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.
[0078] The above are only some embodiments of the present invention. For those skilled in the art, several modifications and improvements can be made without departing from the inventive concept of the present invention, which all fall within the scope of protection of the present invention.
Claims
1. A method for generating a base model pool based on multi-objective evolutionary optimization, characterized by: The following steps are involved: Step 1: Sample data preprocessing: Obtain sample data and preprocess the sample data into the mode required by the algorithm; the sample data is real data in actual application, specifically (Ce0.4Nd0.6)32.7FeB and Nd2Fe14B material data, which includes temperature, deformation speed, deformation amount, coercive force H cj and remanence B r , where temperature, deformation speed and deformation amount are used as input data of the model, and coercive force H cj and remanence B r Two data are used as the output of the model; Step 2: Sample data division: Divide the preprocessed sample data into training set T, validation set V and test set M; Step 3: Base model structure selection: Determine the type and hyperparameters of the base model required for ensemble learning based on the ensemble model to be built; Step 4: Base model pool training: Use a multi-objective evolutionary optimization method to learn the training set T and obtain a set of non-dominant data as the base model S for selection; specifically: Step 4.
1. Initialize base models: Based on the base model types and hyperparameters determined in step 3, initialize a set of base models and determine the mapping rules from the base model hyperparameters to the solutions of the multi-objective optimization method. Step 4.2: Base model fitness evaluation: Input the training set T into the base model obtained in step 4.1, and calculate the fitness value of the base model according to the optimization goal of the preset multi-objective optimization problem; Step 4.3: Base model evolution iteration: Optimize the hyperparameters of each base model through evolutionary operator iteration to promote the evolution of the base model. Step 4.4, Base Model Pool Management: Using the base model hyperparameters obtained in step 4.3, solve the Pareto threshold frontier S for the required base model generation problem. If the preset stopping condition of the multi-objective evolutionary optimization method is reached, the experiment is stopped. The Pareto threshold frontier S at this time is the solution set of the selected optimal base model. Otherwise, continue with steps 4.2 to 4.
4. Step 5: Base model selection: Use the validation set V as the input of the selected base model S obtained in step 4, obtain the output set corresponding to the validation set V, and use this output set and the true output of the validation set V to calculate the mean square error (MSE) of the base model; then, combine the specified rules to select a specified number of base models from the selected base model S obtained in step 4; Step 6: Use the base model obtained in step 5 as the base model for ensemble learning and construct the ensemble model according to the parallel ensemble method.
2. The method for generating a base model pool based on multi-objective evolutionary optimization according to claim 1, wherein: The above-mentioned method for generating a base model pool in ensemble learning further includes: step 7, evaluating the generalization performance of the ensemble model, and the specific evaluation method is: Input the test set M into the integrated model obtained in step 6 and obtain an output set through calculation. Calculate the mean square error (MSE) between the output set obtained by the integrated model and the true output set corresponding to the test set M to evaluate the generalization performance of the integrated model. The smaller the mean square error, the stronger the generalization performance of the integrated model.
3. The method for generating a base model pool based on multi-objective evolutionary optimization according to claim 1, wherein: The preprocessing of the sample data in step 1 is achieved by filtering, denoising, whitening or normalization.
4. The method for generating a base model pool based on multi-objective evolutionary optimization according to claim 1, wherein: The multi-objective optimization method used in step 4 is a multi-objective genetic algorithm, a multi-objective particle swarm optimization algorithm or a multi-objective evolutionary optimization algorithm.
5. The method for generating a base model pool based on multi-objective evolutionary optimization according to claim 1, wherein: The optimization objectives of the multi-objective optimization generation problem preset in step 4.2 include: minimizing the prediction error of each base model in the multi-objective optimization and minimizing the vector norm of the parameters of each base model; wherein the minimization of the prediction error of each base model in the multi-objective optimization represents the generalization performance target of the base model, and the minimization of the vector norm of the parameters of each base model represents the structural simplicity target; the prediction error includes minimizing the prediction mean square error MSE, minimizing the prediction mean absolute error MAE and minimizing the prediction root mean square error RMSE; the vector norm includes minimizing the 1-norm ||w||1 and minimizing the 2-norm ||w||2.
6. The method for generating a base model pool based on multi-objective evolutionary optimization according to claim 1, wherein: When the mean square error MSE set of the base models in step 5 specifies the rule for selecting the base models, the specified rule adopted is to select several base models with smaller prediction errors or base models located at the inflection point on the Pareto threshold frontier.
7. The method for generating a base model pool based on multi-objective evolutionary optimization according to claim 1, characterized in that: When dividing the sample data in step 2, it is also possible to adopt a method of combining multiple divisions of the data to obtain multiple base models with a multi-objective optimization method. By combining the synergy of multiple divisions of the data to obtain multiple base models with the multi-objective optimization method, the generalization performance of the integrated model can be further improved.