Large-scale multi-objective optimization algorithm selection method and system based on depth meta-features

Abstract

The invention discloses a large-scale multi-objective optimization algorithm selection method and system based on depth meta-features. The method comprises the steps of obtaining a problem database; wherein the problem database comprises a plurality of multi-objective optimization problems; selecting a model based on a large-scale multi-objective optimization algorithm, and extracting to obtain dimensionality-reduced features; constructing an algorithm library, and measuring the solving result of each algorithm in the algorithm library on each large-scale multi-objective optimization problem to obtain an algorithm performance sequence corresponding to each algorithm; determining a mapping relation between the problem and the algorithm performance according to the dimension-reduced features and the algorithm performance sequence; and inputting the depth meta-features of the test data into the trained base learner, mapping the depth meta-features to corresponding performance measures according to the mapping relationship, and outputting a performance sequence of the to-be-predicted algorithm. According to the method, the deep meta-features of the large-scale multi-objective optimization problem are extracted in a neural network mode, the algorithm selection accuracy is improved, and the calculation cost of manual meta-features is reduced.

Description

Technical Field

[0001] The present invention relates to the field of algorithm selection technology, and in particular to a large-scale multi-objective optimization algorithm selection method and system based on deep meta-features. Background Art

[0002] Multi-objective optimization (MOP) involves simultaneously optimizing two or more objective functions under specific constraints. It is a common problem in fields such as engineering, economics, and management. These objectives often conflict, making it impossible to achieve optimal results for all objectives simultaneously through a single solution. This characteristic of multi-objective optimization often requires finding a set of compromise solutions, known as Pareto optimal solutions. The hyperplane formed by their spatial distribution is called the Pareto front. Each solution in the Pareto optimal set cannot be further improved on certain objectives without sacrificing performance on other objectives. Therefore, the goal of multi-objective optimization is to find a set of solutions that optimizes each objective as much as possible by balancing the relationships between different objectives.

[0003] With the increasing complexity of multi-objective optimization problems and the development of evolutionary multi-objective optimization algorithms, many high-performance algorithms have emerged to solve multi-objective optimization problems in complex scenarios. When faced with specific multi-objective optimization problems and numerous evolutionary multi-objective optimization algorithms, due to the varying structural characteristics of different problems, researchers often rely on past experience or trial and error to determine which algorithm offers the highest accuracy and efficiency for a particular problem. Furthermore, due to inherent differences in algorithms, an algorithm that performs well on one type of problem may not be applicable to others.

[0004] To address this dilemma, algorithm selection (AS) can improve problem solving by selecting appropriate algorithm instances from a pool of candidate algorithms. This research doesn't focus on proposing new algorithms or improving existing ones. Instead, it uses machine learning to find mappings between problems and algorithms, addressing the limitations of previous researchers who focused solely on innovative algorithms.

[0005] Automated machine learning has been a key research direction in the field of machine learning in recent years. It aims to reduce manual intervention through automation, reduce the complexity of the machine learning process, and make it more user-friendly. This research focuses on data preprocessing, feature engineering, algorithm selection, and hyperparameter optimization. Meta-learning-based algorithm selection relies on meta-features of problem instances. These meta-features effectively characterize the structural characteristics of problem instances, enabling algorithm selection for specific problems.

[0006] In recent years, scholars have proposed effective meta-features for algorithm selection in single-objective optimization and multi-objective optimization fields, and verified the effectiveness of the algorithm selection model they proposed in a specific set of problem instances. However, existing meta-features are mostly selected based on expert experience and are suitable for low-dimensional problems, but they often only describe local characteristics of the problem and lack generalizability, making it difficult to deal with complex high-dimensional scenarios. In high-dimensional problems, such features may not fully capture the complexity of the problem space, affecting the accuracy of algorithm selection. Secondly, hand-crafted meta-features have the problem of feature redundancy. Early methods usually evaluate by combining different types of meta-features, but this strategy fails to break through the feature sensitivity problem. In addition, in terms of the computational cost of meta-features, hand-crafted meta-features are calculated through mathematical expressions, and when the dimension of the problem's decision variable increases, the computational cost rises sharply.

[0007] Overall, current algorithm selection methods still need to break through the bottleneck of meta-feature design in the context of large-scale multi-objective optimization problems, and cannot accurately select effective models, resulting in high computational cost.

[0008] Therefore, the prior art still needs to be improved. SUMMARY

[0009] The technical problem to be solved by the present application is that, in view of the defects of the prior art, the present application provides a large-scale multi-objective optimization algorithm selection method and system based on deep meta-features, to solve the problems of low algorithm selection accuracy and high computational cost existing in the current algorithm selection methods for the field of multi-objective optimization.

[0010] The technical solution adopted by the present application to solve the technical problem is as follows: In a first aspect, the present application provides a large-scale multi-objective optimization algorithm selection method based on deep meta-features, comprising: obtaining a problem database; wherein the problem database comprises: a plurality of multi-objective optimization problems; extracting reduced features based on a large-scale multi-objective optimization algorithm selection model; constructing an algorithm library and measuring the solution results of each algorithm in the algorithm library on each large-scale multi-objective optimization problem to obtain the algorithm performance order corresponding to each algorithm; determining the mapping relationship between the problem and the algorithm performance according to the reduced features and the algorithm performance order; inputting the deep meta-features of the test data into the trained base learner, and mapping the deep meta-features to the corresponding performance measure according to the mapping relationship, and outputting the performance order of the algorithm to be predicted.

[0011] In one implementation, the problem database is obtained by: selecting a plurality of benchmark functions as a problem library; Sampling each benchmark function to obtain a matrix of corresponding decision variables, and calculating the objective function value corresponding to the matrix; According to the sampled matrix and the corresponding objective function value, the corresponding multi-objective optimization problem is described to obtain the problem database.

[0012] In one implementation, the feature extraction module based on the large-scale multi-objective optimization algorithm selection model extracts the reduced-dimensional features, including: Inputting data from the problem database into the large-scale multi-objective optimization algorithm selection model; Based on a stacked autoencoder, features of corresponding dimensions are extracted from the data of the problem database using a trained hidden layer of preset dimensions to obtain the reduced-dimensional features.

[0013] In one implementation, the algorithm library is constructed, and the solution results of each algorithm in the algorithm library on each large-scale multi-objective optimization problem are measured to obtain the algorithm performance ranking corresponding to each algorithm, including: Building the algorithm library; Based on the PlatEMO platform, the mean of the inversion generation distance (IGD) in the comprehensive evaluation index is used as the performance indicator of the algorithm to evaluate the solution results of each algorithm in the algorithm library on various large-scale multi-objective optimization problems, and obtain the algorithm performance order corresponding to each algorithm.

[0014] In one implementation, determining a mapping relationship between a problem and algorithm performance based on the reduced-dimensional features and the algorithm performance order includes: Inputting the deep meta-feature and the algorithm performance order obtained by the performance evaluation module into a trained stacked autoencoder for compression to obtain the deep meta-feature; wherein the stacked autoencoder is composed of at least two independently trained hidden layers stacked; A mapping relationship between the problem and algorithm performance is determined based on the deep meta-features.

[0015] In one implementation, the step of inputting the deep meta-features and the algorithm performance order obtained by the performance evaluation module into a trained stacked autoencoder for compression to obtain the deep meta-features includes: Determine a source domain and a target domain; wherein the source domain is composed of a set of training samples of multiple benchmark functions; and the target domain is a single test function reserved by the leave-one-out method; A domain adaptation method based on maximum mean difference (MMD) is used to reduce the distribution difference between the source domain and the target domain in the feature space, thereby obtaining a trained stacked autoencoder.

[0016] In one implementation, inputting the deep meta-features of the test data into the trained base learner, mapping the deep meta-features to corresponding performance measures according to the mapping relationship, and outputting the performance ranking of the algorithms to be predicted includes: Input the deep meta-features of several training data into the ANN learner and the RF learner for training to obtain the trained base learner; The deep meta-features of the test data are input into the trained base learner, and the performance order of the algorithm to be predicted is output.

[0017] In a second aspect, the present invention provides a large-scale multi-objective optimization algorithm selection system based on deep meta-features, comprising: A database acquisition module is used to acquire a problem database; wherein the problem database includes: a plurality of multi-objective optimization problems; Feature extraction module, used to extract features after dimensionality reduction based on the large-scale multi-objective optimization algorithm selection model; A performance evaluation module is used to build an algorithm library and measure the solution results of each algorithm in the algorithm library on various large-scale multi-objective optimization problems to obtain the algorithm performance ranking corresponding to each algorithm; A meta-learning module, configured to determine a mapping relationship between a problem and an algorithm performance based on the reduced-dimensionality features and the algorithm performance ranking; The algorithm selection module is used to input the deep meta-features of the test data into the trained base learner, and map the deep meta-features to the corresponding performance measures according to the mapping relationship, and output the performance order of the algorithm to be predicted.

[0018] In a third aspect, the present invention provides a terminal comprising: a processor and a memory, wherein the memory stores a large-scale multi-objective optimization algorithm selection program based on deep meta-features, and the large-scale multi-objective optimization algorithm selection program based on deep meta-features is used to implement the operation of the large-scale multi-objective optimization algorithm selection method based on deep meta-features as described in the first aspect when executed by the processor.

[0019] In a fourth aspect, the present invention also provides a medium, which is a computer-readable storage medium, and the medium stores a large-scale multi-objective optimization algorithm selection program based on deep meta-features. When the large-scale multi-objective optimization algorithm selection program based on deep meta-features is executed by a processor, it is used to implement the operation of the large-scale multi-objective optimization algorithm selection method based on deep meta-features as described in the first aspect.

[0020] The present invention adopts the above technical solution to achieve the following effects: The application extracts the reduced features based on a large-scale multi-objective optimization algorithm selection model; constructs an algorithm library, and measures the solving results of each algorithm in each large-scale multi-objective optimization problem in the algorithm library to obtain the algorithm performance order corresponding to each algorithm; determines the mapping relationship between the problem and the algorithm performance according to the reduced features and the algorithm performance order; inputs the deep meta features of the test data into the trained base learner, and maps the deep meta features to the corresponding performance measure according to the mapping relationship to output the performance order of the algorithm to be predicted. The application extracts the deep meta features of the large-scale multi-objective optimization problem in the manner of neural network, improves the algorithm selection accuracy, and reduces the calculation cost of the manual meta features. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained according to the structures shown in the drawings without creative labor for those skilled in the art.

[0022] Figure 1 It is a flow chart of the large-scale multi-objective optimization algorithm selection method based on deep meta features in the present application.

[0023] Figure 2 It is a framework schematic diagram of the large-scale multi-objective optimization algorithm selection in the present application.

[0024] Figure 3 It is a sampling schematic diagram of a single benchmark function in the present application.

[0025] Figure 4 It is a stacked auto-encoder schematic diagram in the present application.

[0026] Figure 5 It is a schematic diagram of the stacking of the hidden layer and the conversion process of the original data to the F features in the present application.

[0027] Figure 6 It is a network structure schematic diagram adopted by the meta learning module in the present application.

[0028] Figure 7 It is a code schematic diagram of the hidden layer training process of the meta learning module in the present application.

[0029] Figure 8 It is a code schematic diagram of the network structure of the meta learning module in the present application.

[0030] Figure 9 It is a model training flow chart after the fusion domain adaptation method in the present application.

[0031] Figure 10 It is a training logic diagram of the entire algorithm selection model in the present invention.

[0032] Figure 11 It is a functional principle diagram of a terminal in one implementation of the present invention.

[0033] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solutions and advantages of the present invention more clear and distinct, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0035] Exemplary Methods Automated machine learning has been a key research direction in the field of machine learning in recent years. It aims to reduce manual intervention through automation, reduce the complexity of the machine learning process, and make it more user-friendly. This research focuses on data preprocessing, feature engineering, algorithm selection, and hyperparameter optimization. Meta-learning-based algorithm selection relies on meta-features of problem instances. These meta-features effectively characterize the structural characteristics of problem instances, enabling algorithm selection for specific problems.

[0036] In response to the problem of a large number of meta-features lacking a unified description, some studies have systematically classified meta-features into the following categories: (1) Simple meta-features, which represent the basic information of a dataset and have low computational cost. Typical features include the number of attributes, the number of binary attributes, and the number of categorical attributes. (2) Statistical meta-features, which describe data from the perspective of various statistical metrics, such as the mean, standard deviation, and correlation of numerical attributes. (3) Meta-features based on information theory, which mainly use information entropy to obtain the amount of information in the data and its complexity. (4) Model-based meta-features, which use machine learning models to extract meta-features of data. For example, features extracted by a decision tree model are also called decision tree-based meta-features. (5) Landmark meta-features, which use the performance of the algorithm on the problem dataset as meta-features.

[0037] In recent years, some researchers have proposed effective meta-features for algorithm selection in single-objective and multi-objective optimization and verified the effectiveness of their proposed algorithm selection models on specific problem instances. For example, some researchers have proposed an adaptive algorithm recommendation model ARM for population-based optimization algorithms. To comprehensively characterize the problem, the model considers three types of meta-features. The statistical features include mean, standard deviation, skewness, kurtosis, search space height, first quartile, median, and third quartile. The geometric features describe the complexity of the problem surface and include gradient features, outlier ratio, and extreme value ratio. The computationally efficient landscape features are used to measure the dispersion of the population and the correlation of the fitness distance. Through the characterization of the target optimization problem and experiments on benchmark functions, they verified the effectiveness of the proposed model in scheduling particle swarm optimization algorithms and their variants.

[0038] In the study of multi-objective optimization problems, some researchers have extended two types of meta-features for multi-objective optimization problems in addition to statistical meta-features, geometric meta-features, and landmark meta-features. The meta-features based on the fitness space measure multi-objective optimization problems from the aspects of multi-modality and uniformity of distribution in the fitness space. The meta-features based on the Pareto front measure multi-objective optimization problems from the aspects of the shape of the Pareto front, continuity, and consistency of the mapping dimension. Through experimental analysis of 70 benchmark functions, the effectiveness of the proposed model MOARM in algorithm selection is confirmed.

[0039] Algorithm selection based on meta-learning has been verified in multiple real-world scenarios, and many researchers have proposed meta-features with good representation ability for complex optimization problems. However, in the context of the curse of dimensionality, existing methods still have limitations in algorithm selection for high-dimensional multi-objective optimization problems. The curse of dimensionality refers to the problem that traditional algorithms perform poorly and the computational cost increases dramatically when dealing with high-dimensional data. This is particularly prominent in large-scale multi-objective optimization problems, making traditional hand-designed meta-feature methods challenging.

[0040] Firstly, existing meta-features are mostly selected based on expert experience and are suitable for low-dimensional problems, but they often only describe local characteristics of the problem, lack generalizability, and are difficult to cope with complex high-dimensional scenarios. In high-dimensional problems, such features may not be able to fully capture the complexity of the problem space, affecting the accuracy of algorithm selection. Secondly, hand-designed meta-features have the problem of feature redundancy. Early methods usually evaluate by combining different types of meta-features, but this strategy fails to overcome the feature sensitivity problem. In addition, in terms of computational cost, hand-designed meta-features are calculated through mathematical expressions, and when the dimension of the problem's decision variables increases, the computational cost rises sharply.

[0041] In general, the current algorithm selection method still needs to break through the bottleneck of meta-feature design in the scenario of large-scale multi-objective optimization problems, so as to reduce feature redundancy and computational cost and more effectively deal with the challenges brought by the "curse of dimensionality". That is, the current algorithm selection method still needs to break through the bottleneck of meta-feature design in the scenario of large-scale multi-objective optimization problems, and cannot accurately select effective models, resulting in high computational costs.

[0042] In response to the above technical problems, an embodiment of the present invention provides a large-scale multi-objective optimization algorithm selection method based on deep meta-features. The method extracts reduced-dimensional features based on a large-scale multi-objective optimization algorithm selection model; constructs an algorithm library, and measures the solution results of each algorithm in the algorithm library on each large-scale multi-objective optimization problem to obtain the algorithm performance order corresponding to each algorithm; determines the mapping relationship between the problem and the algorithm performance based on the reduced-dimensional features and the algorithm performance order; inputs the deep meta-features of the test data into the trained base learner, and maps the deep meta-features to the corresponding performance measures based on the mapping relationship, and outputs the performance order of the algorithm to be predicted. In the embodiment of the present invention, the deep meta-features of large-scale multi-objective optimization problems are extracted using a neural network, which improves the accuracy of algorithm selection and reduces the computational cost of manual meta-features.

[0043] like Figure 1 As shown, an embodiment of the present invention provides a large-scale multi-objective optimization algorithm selection method based on deep meta-features, comprising the following steps: Step S100, obtaining a problem database; wherein the problem database includes: a plurality of multi-objective optimization problems.

[0044] As the number of algorithms increases and problem instances diversify, a single algorithm can no longer provide the optimal solution for all complex optimization scenarios. Different algorithms vary in performance and resource consumption for the same problem, and ordinary users often lack domain knowledge, making finding the optimal algorithm for a specific problem a challenge. Previous research often relied on prior knowledge or trial-and-error methods to find the optimal solution, ignoring the relationship between problem characteristics and algorithm performance, making optimal algorithm selection inefficient.

[0045] The idea of ​​algorithm selection was born as early as the last century. Researchers have achieved algorithm selection in specific fields through the study of problems and algorithms. In the field of multi-objective optimization, existing research has used manual meta-features to achieve algorithm selection for multi-objective optimization problems in the case of low-dimensional decision variables. In order to improve the efficiency of solving large-scale multi-objective optimization problems, the embodiment of the present invention is inspired by the data-driven idea and proposes a large-scale multi-objective optimization model based on deep meta-features. In response to the problem of extremely high dimensionality of decision variables in large-scale situations, this embodiment uses a stacked autoencoder encoder-decoder network to extract deep meta-features of the problem to avoid the feature redundancy and computational cost problems brought about by manual meta-features in previous multi-objective optimization problems, so as to further guide low-level algorithm models to make decisions quickly. Taking into account the problem of domain drift, this embodiment introduces the maximum mean difference (MMD) in the loss function to reduce the distribution difference between training data (source domain) and test data (target domain), thereby improving the generalization of the model on unknown problems.

[0046] In this embodiment, different algorithms are combined and their advantages in different problems are fully utilized to improve the efficiency of solving the problem. Therefore, this embodiment proposes a framework for large-scale multi-objective optimization algorithm selection. The structure of the framework is as follows: Figure 2 As shown, it includes: feature extraction module, performance evaluation module, meta-learning module and algorithm selection module.

[0047] Specifically, in one implementation of this embodiment, step S100 includes the following steps: Step S101, selecting several benchmark functions as a question library; Step S102: sampling each benchmark function to obtain a matrix of corresponding decision variables, and calculating the objective function value corresponding to the matrix; Step S103: Describe the corresponding multi-objective optimization problem based on the sampled matrix and the corresponding objective function value to obtain the problem database.

[0048] In this embodiment, a stacked autoencoder neural network is used to obtain feature representations for large-scale multi-objective optimization problems. This not only overcomes the computational complexity of calculating features for large-scale multi-objective optimization problems, but also eliminates existing hand-crafted features and avoids human bias. In the problem database, the decision variables of the problem data are obtained using Latin hypercube sampling, and their corresponding function values ​​are calculated using Matlab. A multi-objective optimization problem consists of its decision variables and their corresponding function values. The specific data in the problem database of this embodiment includes 20 (1000-dimensional) multi-objective optimization problems (for example, DTLZ1-7, WFG1-8, ZDT1-4, and ZDT6). In the experiment, these 20 functions were set to 1000 dimensions and sampled.

[0049] To represent a multi-objective optimization problem in a computer, it is necessary to sample discrete points (sampling points) within the multi-objective optimization problem and calculate their objective function values. This embodiment employs Latin Hypercube Sampling (LHS). By uniformly sampling the decision variable space, this method reduces the redundancy of sample points in a specific region and more realistically captures the original form of the optimization problem. LHS divides each dimension into evenly spaced subintervals, ensuring that each subinterval contains only one sample point, resulting in a uniform distribution of samples across all dimensions.

[0050] This embodiment selects 20 benchmark functions as the problem library and uses the Matlab platform to sample each benchmark function one by one to achieve matrix storage of the problem. Assuming that the decision variable of a benchmark function is 1000-dimensional and 1000 points are sampled for the benchmark function, the matrix representing the decision variable of this benchmark function is 1000*1000. The matrix of decision variables and their corresponding target values ​​(one-dimensional vectors) together describe a multi-objective optimization problem. The sampling process of a single benchmark function is as follows: Figure 3 shown.

[0051] like Figure 1 As shown, an embodiment of the present invention provides a large-scale multi-objective optimization algorithm selection method based on deep meta-features, comprising the following steps: Step S200: extracting features after dimensionality reduction based on a large-scale multi-objective optimization algorithm selection model.

[0052] In this embodiment, after the problem database is selected, the multi-objective optimization problem therein is input into the large-scale multi-objective optimization algorithm selection model designed in this embodiment, and the feature extraction module therein is used to extract the reduced-dimensional features.

[0053] Specifically, in one implementation of this embodiment, step S200 includes the following steps: Step S201, inputting the data of the problem database into the large-scale multi-objective optimization algorithm selection model; Step S202: Based on a stacked autoencoder, a trained hidden layer of a preset dimension is used to extract features of corresponding dimensions from the data in the question database to obtain the reduced-dimensional features.

[0054] In this embodiment, in the feature extraction module, the following Figure 4 The stacked autoencoder shown in the figure is used to reduce the dimension of the problem. Since the dimension reduction in this step mainly depends on the function value, the feature after dimension reduction is called F feature. The principle of the stacked autoencoder is as follows: The multi-hidden layer structure gives the stacked autoencoder stronger nonlinear representation capabilities, enabling it to capture more complex data patterns and deep features. Its unique training model also overcomes the problem of feature loss in simple autoencoders to a certain extent. The stacked autoencoder is composed of two or more independent autoencoders, and its training process is as follows: 1) Use the first simple autoencoder to process the raw data and obtain its feature representation.

[0055] 2) The hidden layer features output by the previous layer serve as the input of the next simple autoencoder to further extract deep features. This process is repeated several times until all hidden layers are trained.

[0056] 3) All previously trained hidden layers are stacked and fine-tuned in a supervised manner using the back-propagation algorithm.

[0057] In this embodiment Figure 4 The training process and structure of a stacked autoencoder with three hidden layers are clearly shown in Figure 2.

[0058] like Figure 5 As shown, Figure 5 The stacking of hidden layers and the conversion of raw data to its F features are shown in Figure 1. First, two hidden layers are trained separately, then the two hidden layers are stacked and fine-tuned using the raw data, and finally the F features of the raw data are output at the layer of the specified dimension.

[0059] The specific extraction process of F feature is as follows: The dimension of the problem data from the original sampling point to the F feature is reduced from 1000 dimensions to 200 dimensions. The network used in the feature extraction module in this embodiment is a fully connected network with 1000 input nodes (the same as the dimension of the original sample point) and an output layer dimension of 2 (each input sample point needs to be converted into two objective function values). and is the supervisory signal).

[0060] For the hidden layers, two separate networks are first trained using parameter settings with dimensions varying from (1000, 500, 2) to (500, 200, 2). The goal is to obtain weights for the 500-dimensional and 200-dimensional hidden layers. The 500-dimensional input of the latter network comes from the output of the previously trained hidden layer (500-dimensional) after the original data has passed through it. After training the two hidden layers, they are stacked into a (1000, 500, 200, 2) network. This network is then fine-tuned for 10 epochs using the original data (i.e., the neural network fully processes the entire training data during training), resulting in a new network with new weights. Finally, the original data is fed into the (1000, 500, 200, 2) network, and the F feature extracted from the 200-dimensional layer is used as the feature.

[0061] Unlike conventional autoencoders, this example does not utilize the data itself for self-supervised training. Instead, it incorporates supervised learning into the original autoencoder, using two decision variables as labeled data for training. The problem library includes common multi-objective optimization problems. This example uses 20 classic large-scale multi-objective optimization problems (DTLZ1-7, WFG1-8, ZDT1-4, and ZDT6) as benchmark functions to enrich the problem space.

[0062] Two decision variables are the function values ​​corresponding to a high-dimensional sampling point. Each multi-objective optimization sample is 1000 sampling points, and each point calculates two scalars through the expression of the multi-objective optimization problem itself. and In the process of F feature extraction, the neural network input is the one-dimensional vector of the sample point, and the corresponding supervisory signal is and .

[0063] In this embodiment, the stacked autoencoder of the feature extraction module is used to reduce the dimension of the problem, and the F features of the corresponding dimensions are extracted from the data of the problem database using the trained hidden layer of the preset dimension, so that the mapping relationship between the problem and the algorithm performance can be learned by using the F features and the algorithm performance order obtained by the performance evaluation module.

[0064] like Figure 1 As shown, an embodiment of the present invention provides a large-scale multi-objective optimization algorithm selection method based on deep meta-features, comprising the following steps: Step S300: construct an algorithm library, and measure the solution results of each algorithm in the algorithm library on each large-scale multi-objective optimization problem to obtain the algorithm performance order corresponding to each algorithm.

[0065] In this embodiment, when constructing the algorithm library in the performance evaluation module, the selected algorithms should be representative. It is necessary to ensure that the algorithms can solve all problems in the problem library and at the same time meet the requirement of "complementarity". An algorithm cannot perform too well on most problems and must meet the "No Free Lunch" assumption (the theory of no free lunch in machine learning).

[0066] Specifically, in one implementation of this embodiment, step S300 includes the following steps: Step S301, constructing the algorithm library; Step S302: Based on the PlatEMO platform, the average value of the inversion generation distance (IGD) in the comprehensive evaluation index is used as the performance indicator of the algorithm to evaluate the solution results of each algorithm in the algorithm library on each large-scale multi-objective optimization problem, and obtain the performance ranking corresponding to each algorithm.

[0067] In this embodiment, six representative evolutionary multi-objective optimization algorithms are selected as the algorithm library, namely MOEA / D algorithm, NSGAII algorithm, NSGAIII algorithm, IBEA algorithm, SPEA2 algorithm and eMOEA algorithm.

[0068] MOEA / D (Multi-Objective Evolutionary Algorithm based on Decomposition) is a decomposition-based multi-objective evolutionary algorithm. It transforms a multi-objective problem into a series of single-objective subproblems and optimizes these subproblems using evolutionary operators (such as crossover and mutation) and a neighbor search strategy. MOEA / D achieves population diversity and convergence by performing decomposition in different regions of the solution space.

[0069] NSGA-II (Non-dominated Sorting Genetic Algorithm II) is a classic multi-objective evolutionary algorithm. It maintains population diversity based on non-dominated sorting and crowding distance. NSGA-II uses a fast non-dominated sorting algorithm to classify individuals into non-dominated levels and uses crowding distance to select solutions with good fitness and diversity.

[0070] NSGA-III (Non-dominated Sorting Genetic Algorithm III) is an extension of NSGA-II, designed to solve high-dimensional multi-objective optimization problems. NSGA-III maintains population diversity by introducing reference points and allocation strategies. It uses non-dominated sorting and crowding distance to select solutions, and uses reference points to guide the distribution of solutions, ultimately achieving a uniformly distributed Pareto-optimal set of solutions.

[0071] IBEA (Indicator-Based Evolutionary Algorithm) is an indicator-based, multi-objective evolutionary algorithm. It uses an indicator function to measure the quality and diversity of solutions and selects excellent solutions based on these indicators. The IBEA algorithm optimizes the indicator function to find Pareto-optimal solutions and maintains population diversity during the evolutionary process.

[0072] SPEA2 (Strength Pareto Evolutionary Algorithm 2) is a strength-based, multi-objective evolutionary algorithm. It uses non-dominated sorting and individual crowding to calculate individual strengths and updates the population by selecting and retaining individuals with higher strengths. SPEA2 maintains an external archive to preserve non-dominated solutions and maintains diversity through crowding calculations.

[0073] eMOEA is a multi-objective evolutionary algorithm based on the ε-dominance relationship. It uses the ε-dominance relationship to determine the relative merits of solutions and adjusts the diversity and convergence of solutions by controlling the ε value. The eMOEA algorithm introduces a set of ε values ​​into the solution space to capture the Pareto optimal solution set, thereby achieving a non-uniform distribution of solutions.

[0074] The evolutionary multi-objective optimization algorithms selected in this embodiment have extensive applications and research in solving multi-objective optimization problems. Each algorithm has its own unique characteristics and advantages, and is applicable to different types of multi-objective problems. Researchers can select the appropriate algorithm to solve the problem based on the characteristics and requirements of the problem.

[0075] The performance evaluation module in this embodiment is used to measure the effectiveness of each algorithm in the algorithm library in solving various large-scale multi-objective optimization problems. Based on the characteristics of the Pareto optimal surface, there are roughly three types of performance evaluation methods for existing evolutionary multi-objective optimization algorithms: 1) From the perspective of convergence, evaluate the degree of convergence between the solution set and the true Pareto frontier; 2) From the perspective of distribution, evaluate the uniformity of non-dominated individuals in the solution set in the entire solution space; 3) Comprehensive evaluation indicators are used to simultaneously reflect the convergence and distribution of the solution.

[0076] In this embodiment, the inversion generation distance in the comprehensive evaluation index is used as the performance index of the algorithm, and the computing platform adopts PlatEMO. Other types of performance indexes can also be adopted.

[0077] The actual performance of the algorithms in this example serves as a label for the machine learning task. Therefore, the actual performance of the algorithms in this example is directly calculated using PlatEMO. Using the inverse generation distance (IGD), a common comprehensive performance evaluation metric, the performance of the algorithms can be determined by selecting the problem to be evaluated (for example, by setting the number of bits and sample points) and several algorithms on the PlatEMO platform. By comparing the performance metrics of the algorithms calculated by the platform, the performance ranking of the algorithms can be determined.

[0078] like Figure 1 As shown, an embodiment of the present invention provides a large-scale multi-objective optimization algorithm selection method based on deep meta-features, comprising the following steps: Step S400: determining a mapping relationship between the problem and the algorithm performance according to the features after dimensionality reduction and the algorithm performance order.

[0079] In this embodiment, the feature extraction module compresses the decision variables and their corresponding target values ​​of the multi-objective optimization problem into F features using a stacked autoencoder. The meta-learning module in this embodiment uses the F features extracted by the feature extraction module and the algorithm performance ranking obtained by the performance evaluation module to learn the mapping relationship between the problem and algorithm performance.

[0080] The meta-learning module in this embodiment further compresses the F feature through the stacked autoencoder to obtain the deep meta-feature. The learning of this deep meta-feature depends on the algorithm performance, so it is called P feature. The stacked autoencoder used in this stage undertakes the important task of extracting deep meta-features, so it is called meta-learner in this embodiment. Figure 6 As shown, Figure 6 The network structure adopted by this meta-learning module is shown in Figure .

[0081] Specifically, in one implementation of this embodiment, step S400 includes the following steps: Step S401: Inputting the deep meta-features and the algorithm performance order obtained by the performance evaluation module into a trained stacked autoencoder for compression to obtain deep meta-features; wherein the stacked autoencoder is composed of at least two independently trained hidden layers stacked together; Step S402: Determine the mapping relationship between the problem and algorithm performance based on the depth meta-feature.

[0082] In the embodiment, the inputting the deep meta-feature and the algorithm performance sequence obtained by the performance evaluation module into the trained stacked autoencoder for compression to obtain the deep meta-feature includes: determining a source domain and a target domain; the source domain is composed of a training sample set of a plurality of benchmark functions; the target domain is a single test function reserved by the leave-one-out method; and a domain adaptation method based on maximum mean difference (MMD) is used to reduce the distribution difference of the source domain and the target domain in the feature space to obtain the trained stacked autoencoder.

[0083] In the embodiment, the specific extraction process of the P feature is as follows: In the research framework of the algorithm selection problem, the benchmark functions are generated by sampling mathematical expressions. The sampling points of the same benchmark function follow a specific probability distribution, and the sample sets between different benchmark functions have a distribution shift phenomenon. This distribution heterogeneity is reflected not only in the dimension difference of the feature space, but also in the morphological diversity of the objective function. Although there are differences in distribution, the goal of machine learning is to fit or predict the performance sequence of the candidate algorithm. Based on the distribution shift phenomenon between different benchmark functions, the extraction of the deep meta-feature in the embodiment is regarded as a domain adaptation problem, and the problem of poor generalization on the test problem is solved by reducing the distance between the domains or by jointly pulling the source domain and the target domain to the hidden layer space with the same feature distribution.

[0084] In the leave-one-out cross-validation scenario, the source domain is composed of a training sample set of a plurality of benchmark functions (including 20 benchmark functions (DTLZ1-7, WFG1-8, ZDT1-4 and ZDT6)), and the target domain is a single test function reserved by the leave-one-out method. In order to improve the performance of the model on the target domain, the domain adaptation scheme based on maximum mean difference (MMD) is used to reduce the distribution difference of the source domain and the target domain in the feature space in the embodiment, so as to improve the generalization performance of the model on the unseen test function. In the training process, MMD is used as a regularization term together with the task loss MSE to form an optimization loss function. Through the joint guidance of the two losses, the model not only performs well on the source domain, but also generalizes better to the target domain by reducing the distribution difference, solves the distribution mismatch problem between the source domain function and the target domain function caused by the domain drift, and finally realizes the improvement of the generalization ability of the model. This method can also be extended to other similar scenarios, such as cross-device, cross-language or cross-environment tasks.

[0085] As shown in Figure 9 , the P feature is extracted from the performance sequence of the algorithm. Figure 9The model training process after integrating domain adaptation methods is shown in Figure 2. Traditional machine learning processes often focus only on learning the model on the training data. In the field of domain adaptation, the training phase often combines the features of cross-domain data to align the feature space. In the training phase of the meta-learning module, whether training a single hidden layer or a stacked hidden layer, MMD is involved and both are used. Figure 9 The structure shown. Figure 9 As shown in Figure 2, the network input layer consists of two parallel branches: the green branch receives the F feature vector of the training benchmark function, and the blue branch independently processes the F feature vector of the test function. This dual-channel design allows the model to explicitly capture the distribution differences between the source domain (the set of training benchmark functions) and the target domain (the test function). During the feature processing stage, the inputs of the two branches are compressed through a shared hidden layer with a nonlinear activation function. In this embodiment, the feature compression process is implemented using a multilayer perceptron, whose parameters are driven by a joint optimization objective.

[0086] As an example, the P feature extraction method in the actual application scenario of this embodiment is: The learning process from F features to P features occurs in the meta-learning module. The data is the 200-dimensional F features, and the label is the average IGD calculated by the performance evaluation module. The training logic is exactly the same as that for extracting F features.

[0087] The dimension changes to (200, 100, 30, 6), where 6 means that the label part needs to fit the IGD values ​​of 6 algorithms.

[0088] Furthermore, the usage process of the meta-learning module MMD is as follows: First, the meta-learning module trains two hidden layers separately. The dimensions of these two networks are changed to (200, 100, 6) and (100, 30, 6). Then, they are stacked and trained as (200, 100, 30, 6). During this period, the MMD loss is calculated in the three hidden layers during training.

[0089] Specifically, during the training process, the network inputs the data to be predicted (target domain data) into the network, calculates the loss through MMD at 100 dimensions and 30 dimensions, and then adds it to the MSE loss as the total loss.

[0090] Among them, the code of the hidden layer training process of the meta-learning module is as follows Figure 7 As shown, the code of the network model of the meta-learning module is as follows Figure 8 shown.

[0091] like Figure 1 As shown, an embodiment of the present invention provides a large-scale multi-objective optimization algorithm selection method based on deep meta-features, comprising the following steps: Step S500, input the deep meta features of the test data into the trained base learner, and map the deep meta features to the corresponding performance measure according to the mapping relationship, and output the performance order of the algorithm to be predicted.

[0092] In the present embodiment, in the first three modules, the original data is compressed into F features by the feature extraction module, and the F features and the corresponding performance measure are compressed into P features by the meta learning module. At this point, the large-scale multi-objective optimization problem has been greatly dimensionally reduced. The meta learner has fully learned the mapping relationship between the problem and the performance measure. In the last algorithm selection module, only a simple base learner is needed to further learn the mapping of the P features to the corresponding performance measure.

[0093] Specifically, in an implementation manner of the present embodiment, step S500 includes the following steps: Step S501, input the deep meta features of a plurality of training data into the ANN learner and the RF learner for training, to obtain a trained base learner; Step S502, input the deep meta features of the test data into the trained base learner, and output the performance order of the algorithm to be predicted.

[0094] In the present embodiment, in the last algorithm selection module, only a simple base learner is needed to further learn the mapping of the P features to the corresponding performance measure, and finally the P features of the test data are input into the trained base learner, so as to output the performance order of the algorithm to be predicted.

[0095] The data in the present embodiment has been greatly dimensionally reduced from the shape of the original sampling point to the P features. A base function is reduced from 1000*1000 to 1000*30 (30-dimensional features, i.e. the "deep meta features" in the present embodiment). At this time, the deep meta features of 19 training data are input into the ANN and RF learners for training, and then the deep meta features of the problem to be predicted are input into the learner for prediction. The ANN learner adopts a fully connected neural network, and the dimension changes to 30, 12 and 6. The RF learner adopts the RandomForestRegressor tool, and the parameters are default parameters. Since KNN is an instance-based learning algorithm (lazy learning), it predicts by the nearest k samples of the sample to be confirmed. In the present embodiment, KneighborsRegressor is adopted.

[0096] In the present embodiment, the P features of the test data are input into the trained base learner, so as to obtain the final algorithm performance to be predicted. In summary, the training logic of the entire algorithm selection model has been described in detail in the present embodiment, as shown in Figure 10In the base learner, this embodiment selects the three most common meta-algorithms in algorithm selection, KNN, ANN, and RF, as the final prediction algorithms.

[0097] This embodiment achieves the following technical effects through the above technical solution: 1) To avoid the feature redundancy and computational cost issues associated with manual meta-features in previous multi-objective optimization problems, this embodiment uses an encoder-decoder network of a stacked autoencoder to extract deep meta-features of the problem. This further guides lower-level algorithm models to make quick decisions, improving algorithm selection accuracy and reducing the computational cost of manual meta-features.

[0098] 2) Considering the problem of domain drift, this embodiment introduces the maximum mean difference (MMD) into the loss function to narrow the distribution difference between training data (source domain) and test data (target domain), thereby improving the generalization of the model on unknown problems.

[0099] Exemplary devices Based on the above embodiments, the present invention further provides a large-scale multi-objective optimization algorithm selection system based on deep meta-features, comprising: A database acquisition module is used to acquire a problem database; wherein the problem database includes: a plurality of multi-objective optimization problems; Feature extraction module, used to extract features after dimensionality reduction based on the large-scale multi-objective optimization algorithm selection model; A performance evaluation module is used to build an algorithm library and measure the solution results of each algorithm in the algorithm library on various large-scale multi-objective optimization problems to obtain the algorithm performance ranking corresponding to each algorithm; A meta-learning module, configured to determine a mapping relationship between a problem and an algorithm performance based on the reduced-dimensionality features and the algorithm performance ranking; The algorithm selection module is used to input the deep meta-features of the test data into the trained base learner, and map the deep meta-features to the corresponding performance measures according to the mapping relationship, and output the performance order of the algorithm to be predicted.

[0100] This embodiment achieves the following technical effects through the above technical solution: 1) To avoid the feature redundancy and computational cost issues associated with manual meta-features in previous multi-objective optimization problems, this embodiment uses an encoder-decoder network of a stacked autoencoder to extract deep meta-features of the problem. This further guides lower-level algorithm models to make quick decisions, improving algorithm selection accuracy and reducing the computational cost of manual meta-features.

[0101] 2) Considering the problem of domain drift, this embodiment introduces the maximum mean difference (MMD) into the loss function to narrow the distribution difference between training data (source domain) and test data (target domain), thereby improving the generalization of the model on unknown problems.

[0102] Based on the above embodiment, the present invention further provides a terminal, whose principle block diagram can be shown as follows: Figure 11 shown.

[0103] The terminal includes: a processor, memory, interface, display screen and communication module connected via a system bus; wherein the processor of the terminal is used to provide computing and control capabilities; the memory of the terminal includes a storage medium and an internal memory; the storage medium stores an operating system and computer programs; the internal memory provides an environment for the operation of the operating system and computer programs in the storage medium; the interface is used to connect to external devices; the display screen is used to display corresponding information; and the communication module is used to communicate with a cloud server or other devices.

[0104] When the computer program is executed by a processor, it is used to implement the operation of a large-scale multi-objective optimization algorithm selection method based on deep meta-features.

[0105] It will be understood by those skilled in the art that Figure 11 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the solution of the present invention, and does not constitute a limitation on the terminal to which the solution of the present invention is applied. The specific terminal may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0106] In one embodiment, a terminal is provided, which includes: a processor and a memory, wherein the memory stores a large-scale multi-objective optimization algorithm selection program based on deep meta-features, and when the large-scale multi-objective optimization algorithm selection program based on deep meta-features is executed by the processor, it is used to implement the operations of the large-scale multi-objective optimization algorithm selection method based on deep meta-features as described above.

[0107] In one embodiment, a storage medium is provided, wherein the storage medium stores a large-scale multi-objective optimization algorithm selection program based on deep meta-features, and when the large-scale multi-objective optimization algorithm selection program based on deep meta-features is executed by a processor, it is used to implement the operation of the above-mentioned large-scale multi-objective optimization algorithm selection method based on deep meta-features.

[0108] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile storage medium. When executed, the computer program can include the processes in the above-described method embodiments. Any reference to memory, storage, database, or other media used in the various embodiments provided herein may include both non-volatile and volatile memory.

[0109] In summary, the present invention provides a large-scale multi-objective optimization algorithm selection method and system based on deep meta-features, including: obtaining a problem database; wherein the problem database includes: a number of multi-objective optimization problems; extracting reduced-dimensional features based on a large-scale multi-objective optimization algorithm selection model; constructing an algorithm library, and measuring the solution results of each algorithm in the algorithm library on each large-scale multi-objective optimization problem, and obtaining the algorithm performance order corresponding to each algorithm; determining the mapping relationship between the problem and the algorithm performance based on the reduced-dimensional features and the algorithm performance order; inputting the deep meta-features of the test data into the trained base learner, and mapping the deep meta-features to the corresponding performance measures based on the mapping relationship, and outputting the performance order of the algorithm to be predicted. The present invention uses a neural network to extract the deep meta-features of large-scale multi-objective optimization problems, which improves the accuracy of algorithm selection and reduces the computational cost of manual meta-features.

[0110] It should be understood that the application of the present invention is not limited to the above examples. For those skilled in the art, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A large-scale multi-objective optimization algorithm selection method based on deep meta-features, characterized by: include: Obtaining a problem database; wherein the problem database includes: a plurality of multi-objective optimization problems; The features after dimensionality reduction are extracted by selecting the model based on a large-scale multi-objective optimization algorithm; Construct an algorithm library and measure the solution results of each algorithm in the algorithm library on various large-scale multi-objective optimization problems to obtain the algorithm performance ranking corresponding to each algorithm; Determining a mapping relationship between the problem and the algorithm performance according to the features after the dimensionality reduction and the algorithm performance order; The deep meta-features of the test data are input into the trained base learner, and the deep meta-features are mapped to corresponding performance measures according to the mapping relationship, and the performance order of the algorithm to be predicted is output.

2. The method for selecting a large-scale multi-objective optimization algorithm based on deep meta-features according to claim 1, characterized in that: The obtaining of the question database includes: Select several benchmark functions as the problem library; Sampling each benchmark function to obtain a matrix of corresponding decision variables, and calculating the objective function value corresponding to the matrix; According to the sampled matrix and the corresponding objective function value, the corresponding multi-objective optimization problem is described to obtain the problem database.

3. The large-scale multi-objective optimization algorithm selection method based on deep meta-features according to claim 1 is characterized in that: The feature extraction module based on the large-scale multi-objective optimization algorithm selection model extracts the reduced-dimensional features, including: Inputting data from the problem database into the large-scale multi-objective optimization algorithm selection model; Based on a stacked autoencoder, features of corresponding dimensions are extracted from the data of the problem database using a trained hidden layer of preset dimensions to obtain the reduced-dimensional features.

4. The large-scale multi-objective optimization algorithm selection method based on deep meta-features according to claim 1 is characterized in that: The algorithm library is constructed, and the solution results of each algorithm in the algorithm library on each large-scale multi-objective optimization problem are measured to obtain the algorithm performance order corresponding to each algorithm, including: Building the algorithm library; Based on the PlatEMO platform, the mean of the inversion generation distance (IGD) in the comprehensive evaluation index is used as the performance indicator of the algorithm to evaluate the solution results of each algorithm in the algorithm library on various large-scale multi-objective optimization problems, and obtain the algorithm performance order corresponding to each algorithm.

5. The large-scale multi-objective optimization algorithm selection method based on deep meta-features according to claim 1 is characterized in that: Determining a mapping relationship between a problem and algorithm performance according to the features after dimensionality reduction and the algorithm performance order includes: Inputting the deep meta-feature and the algorithm performance order obtained by the performance evaluation module into a trained stacked autoencoder for compression to obtain the deep meta-feature; wherein the stacked autoencoder is composed of at least two independently trained hidden layers stacked; A mapping relationship between the problem and algorithm performance is determined based on the deep meta-features.

6. The method for selecting a large-scale multi-objective optimization algorithm based on deep meta-features according to claim 5, characterized in that: The step of inputting the deep meta-features and the algorithm performance order obtained by the performance evaluation module into the trained stacked autoencoder for compression to obtain the deep meta-features includes: Determine a source domain and a target domain; wherein the source domain is composed of a set of training samples of multiple benchmark functions; and the target domain is a single test function reserved by the leave-one-out method; A domain adaptation method based on maximum mean difference (MMD) is used to reduce the distribution difference between the source domain and the target domain in the feature space, thereby obtaining a trained stacked autoencoder.

7. The method for selecting a large-scale multi-objective optimization algorithm based on deep meta-features according to claim 1, characterized in that: Inputting the deep meta-features of the test data into the trained base learner, mapping the deep meta-features to corresponding performance measures according to the mapping relationship, and outputting the performance order of the algorithm to be predicted includes: Input the deep meta-features of several training data into the ANN learner and the RF learner for training to obtain the trained base learner; The deep meta-features of the test data are input into the trained base learner, and the performance order of the algorithm to be predicted is output.

8. A large-scale multi-objective optimization algorithm selection system based on deep meta-features, characterized by: include: A database acquisition module is used to acquire a problem database; wherein the problem database includes: a plurality of multi-objective optimization problems; Feature extraction module, used to extract features after dimensionality reduction based on the large-scale multi-objective optimization algorithm selection model; A performance evaluation module is used to build an algorithm library and measure the solution results of each algorithm in the algorithm library on various large-scale multi-objective optimization problems to obtain the algorithm performance ranking corresponding to each algorithm; A meta-learning module, configured to determine a mapping relationship between a problem and an algorithm performance based on the reduced-dimensionality features and the algorithm performance ranking; The algorithm selection module is used to input the deep meta-features of the test data into the trained base learner, and map the deep meta-features to the corresponding performance measures according to the mapping relationship, and output the performance order of the algorithm to be predicted.

9. A terminal, characterized in that: include: A processor and a memory, wherein the memory stores a large-scale multi-objective optimization algorithm selection program based on deep meta-features, and when the large-scale multi-objective optimization algorithm selection program based on deep meta-features is executed by the processor, it is used to implement the operation of the large-scale multi-objective optimization algorithm selection method based on deep meta-features as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a large-scale multi-objective optimization algorithm selection program based on deep meta-features, which, when executed by a processor, is used to implement the operation of the large-scale multi-objective optimization algorithm selection method based on deep meta-features as described in any one of claims 1 to 7.

Images