A bi-level optimization method, system and computer storage device therefor for single-objective large-scale expensive optimization problems

Abstract

This invention provides a two-layer optimization method, system, and computer storage device for large-scale, expensive single-objective optimization problems. The method involves steps including initialization, subproblem construction, surrogate model construction, subproblem optimization and evaluation, a bottom-level population update stage, a top-level surrogate model selection stage, and a top-level optimization and global update stage. By constructing and evaluating multiple candidate subspaces at the lower layer, it automatically identifies key subspaces that significantly impact the global objective and focuses on optimizing these key subspaces at the upper layer. Under a finite expensive function evaluation budget, it achieves efficient solutions to large-scale single-objective optimization problems. By introducing a surrogate model to approximate the expensive objective function and combining it with a two-layer optimization structure, it organically combines subspace selection and fine-tuning, effectively improving search efficiency, accelerating convergence speed, and enhancing the quality of the final solution, thus meeting the application needs of complex engineering optimization scenarios.

Description

Technical Field

[0001] This invention relates to the field of optimization computing technology, and in particular to a two-layer optimization method, system, and computer storage device for large-scale, expensive single-objective optimization problems. Background Technology

[0002] With the rapid development of fields such as engineering design optimization, intelligent manufacturing, complex system modeling, and machine learning parameter tuning, large-scale optimization problems (LSOPs) are becoming increasingly prevalent in practical applications. These problems typically exhibit significant characteristics such as high dimensionality of decision variables (e.g., hundreds to thousands of dimensions) and high cost of evaluating the objective function, making it difficult for traditional optimization methods to obtain high-quality solutions with limited computational resources. Existing methods for large-scale optimization problems mainly include direct optimization methods based on evolutionary algorithms and co-evolutionary methods based on decomposition strategies. Co-evolutionary methods typically reduce the search difficulty by dividing the high-dimensional decision space into multiple low-dimensional subspaces. However, existing methods generally assume that each subspace has equal importance, failing to effectively distinguish subspaces that contribute differently to the global objective during actual optimization, easily leading to wasted search resources.

[0003] To reduce the computational overhead of expensive function evaluation, surrogate models (such as radial basis function models and Gaussian process models) are introduced into the optimization process to approximate the true objective function. However, existing surrogate-assisted optimization methods mostly focus on subspace construction itself, lacking sufficient mechanisms for screening and re-optimizing the constructed subspaces. Especially when facing large-scale expensive optimization problems, three technical shortcomings remain: 1. A large number of subspaces but a lack of effective evaluation mechanisms, with important subspaces easily overlooked; 2. The subspace optimization and global optimization processes are not tightly coupled, making it difficult to fully utilize existing search information; 3. A single-level optimization structure cannot simultaneously meet the dual needs of subspace screening and fine-grained optimization. Furthermore, in complex engineering fields such as structural design optimization, aerospace and engine parameter calibration, manufacturing process parameter tuning, power system operation optimization, and complex system parameter configuration, large-scale expensive single-objective optimization problems (LSEOPs) are widespread. These problems typically have characteristics such as high dimensionality of decision variables, high cost of objective function evaluation, and limited computational resources. Existing optimization methods often suffer from the following shortcomings when dealing with the above problems: On the one hand, traditional evolutionary optimization algorithms require a large number of objective function evaluations in high-dimensional search spaces, which makes it difficult to meet the actual needs under expensive evaluation conditions; on the other hand, existing methods based on subspaces or decomposition strategies usually assume that each subspace contributes equally to the global objective and lack an effective mechanism for distinguishing the importance of subspaces, resulting in the inefficient allocation of limited computing resources and affecting optimization performance.

[0004] Therefore, in view of the problems existing in the prior art, it is particularly important to have a two-layer optimization technique that is computationally efficient, highly adaptable, and capable of solving large-scale and expensive single-objective optimization problems with a limited evaluation budget. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a surrogate model-assisted two-layer optimization method, system, and computer storage device for large-scale, expensive single-objective optimization problems. This method automatically identifies key subspaces that significantly impact the global objective by constructing and evaluating multiple candidate subspaces at the lower layer, and then focuses on optimizing these key subspaces at the upper layer. This achieves efficient solutions to large-scale single-objective optimization problems within a finite expensive function evaluation budget. By introducing a surrogate model to approximate the expensive objective function and combining it with a two-layer optimization structure, the method organically combines subspace selection and fine-tuning, effectively improving search efficiency, accelerating convergence speed, and enhancing the quality of the final solution, thus meeting the practical application needs of complex engineering optimization scenarios.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: A bi-level optimization method for large-scale, expensive single-objective optimization problems, the bi-level optimization method comprising: Step S1. Initialization: A set of diverse solutions is generated using Latin hypercube sampling, then evaluated using an expensive true function, and finally stored in the archive set. This collection of archives The training data and optimal solution repository for the surrogate model; and, for those containing Population of individuals Perform initialization processing to provide a search foundation for subsequent optimization; Step S2. Subproblem Construction: Generate subproblems sequentially. There are several subspaces, and the construction process of each subspace is as follows: in the interval... Randomly generate integers Determine the subspace dimension; then randomly select from the original problem decision variables. Each variable constitutes a subproblem The subproblem The decision space is represented as The corresponding temporary population is denoted as ; Step S3. Proxy Model Construction: From the Archives Random selection Extracting samples from sub-problems Based on the data of the corresponding variables, a radial basis surrogate model is trained. (i.e., the RBF model); Step S4. Sub-problem optimization and evaluation: In the first step... Subspace Within the subspace, a differential evolution algorithm is used to iteratively search for candidate solutions. In the population, new individuals are generated based on the DE / best / 1 mutation strategy and the binomial crossover operator; a binomial crossover is performed on the mutation vector and the target vector to generate experimental individuals; and a surrogate model is used for the generated experimental individuals. Predict its fitness value; after a preset number of iterative search rounds, from the subspace The solution with the best predicted fitness is obtained through screening. Its predicted fitness value This reflects the optimization potential of the subspace; by comparing the fitness of the optimal solutions to all subproblems, the current optimal subspace is determined. Step S5. Bottom Population Update Phase: Utilizing the current subproblem The optimization results update the original population The variable values ​​for the corresponding dimension; Step S6. Top-level proxy model selection: For the selected optimal subspace, adaptively select a proxy model based on its dimensional characteristics; Step S7. Top-level optimization and global update stage: The particle swarm optimization algorithm with social learning mechanism is used to perform a fine search in the optimal subspace to obtain the local optimal solution in the subspace. Compare this solution with the optimal solution obtained from the underlying optimization. Combine and jointly update the global optimal solution The variable values ​​for the corresponding dimension.

[0007] Specifically, the above technical solution introduces a surrogate model to predict and evaluate the potential optimization effects of multiple candidate subspaces, constructing an automatic subspace importance identification mechanism. This mechanism can effectively distinguish the contribution of different subspaces to the global objective function based on predicted fitness information, thus prioritizing key subspaces with a greater impact on the optimization results and avoiding wasting limited computing resources on ineffective search processes of low-contribution subspaces. Secondly, this invention adopts a two-layer optimization structure, decoupling the subspace selection and fine-tuning processes. The lower-layer optimization focuses on rapidly constructing and evaluating multiple subspaces in a large-scale decision space to identify the most promising optimization direction; the upper-layer optimization conducts in-depth searches on the selected key subspaces. This hierarchical design makes the optimization process structure clear and its responsibilities well-defined, significantly improving overall search efficiency and accelerating algorithm convergence. Furthermore, this invention adaptively selects different types of surrogate models to approximate expensive objective functions based on subspace dimensional characteristics and search requirements, effectively reducing the computational complexity of surrogate model training and inference while ensuring prediction accuracy, thereby improving the practicality of the method in large-scale scenarios. Furthermore, even with a limited number of function evaluations, this invention can still effectively handle large-scale optimization problems with hundreds or even thousands of decision variables, demonstrating good scalability and stability. It is particularly suitable for engineering optimization scenarios where objective function evaluation is costly. Finally, this invention does not make specific assumptions about the form of the objective function, possessing strong versatility. It can be widely applied to structural design optimization, aerospace parameter calibration, manufacturing process optimization, energy system configuration, and other complex engineering optimization fields and scenarios. It is easily integrated with existing engineering simulation or experimental platforms, enabling its engineering implementation.

[0008] Specifically, this technical solution constructs a two-layer collaborative optimization mechanism during the overall optimization process to achieve hierarchical processing of subspace generation, screening, and fine-tuning. The lower layer constructs and evaluates multiple candidate subspaces in a high-dimensional decision space, while the upper layer performs in-depth searches of selected key subspaces, thereby improving the overall efficiency of large-scale single-objective optimization under a limited and expensive function evaluation budget. A surrogate model is used to predict the objective function values ​​of candidate solutions in different subspaces, and the subspaces are compared and ranked based on the prediction results, thus achieving quantitative evaluation and automatic screening of subspace importance. This mechanism can effectively identify key subspaces that contribute significantly to the improvement of the global objective function, providing a clear and reliable search direction for subsequent optimization processes. Furthermore, by decoupling the subspace screening process from the subspace fine-tuning process in the algorithm structure, a collaborative optimization strategy is formed. Subspace screening focuses on quickly identifying potentially excellent search regions, while fine-tuning focuses on in-depth searches within selected key subspaces. This strategy avoids the efficiency degradation caused by the high coupling of the search process in traditional methods, improving the stability and convergence performance of the algorithm in high-dimensional search spaces.

[0009] Specifically, without departing from the core idea of ​​this invention, the following equivalent substitutions can be made: the method of generating the lower-level subspace can be replaced by randomly selecting variables to a selection method based on correlation analysis of historical data; the subspace importance evaluation index can be replaced by predicting the optimal value to predicting the improvement magnitude, stability index, or a comprehensive evaluation of multiple indicators; the upper-level optimization algorithm can be replaced by particle swarm optimization to differential evolution, evolutionary strategies, or other swarm intelligence algorithms; the surrogate model can be replaced by a support vector regression model, an ensemble learning model, or a multi-model fusion strategy. All of the above substitutions fall within the protection scope of this invention.

[0010] Above, in step S3, the radial basis surrogate model It is a three-layer network structure, which includes an input layer, a hidden layer, and an output layer.

[0011] Above, in step S3, the input point of the input layer The quantity is of Population of decision variables, calculating sample points and sample points The distance between them is calculated as follows: (This distance is the Euclidean distance), forming a matrix Use the cube kernel function to obtain the symmetric matrix of similarity between sample points. The calculation method is as follows: ; Then, use the augmented matrix A to construct a linear system to solve for the parameters. and A is composed of a symmetric matrix and polynomial part Composition; where P is the first-order polynomial extension matrix obtained by adding a constant term 1 to each row of the input point s, and the augmented matrix A is represented as: ; and by solving the system of linear equations To obtain parameters and Right-hand vector It is an input point target value And the zero vector of the tail part of the polynomial: , and They are The former The formula for constructing a surrogate model that can be used for direct prediction is: (The formula is missing from the original text.) ,in, For the polynomial part, yes The last element of the vector, For the new input point, This represents the corresponding prediction result.

[0012] Above, in step S4, in the DE / best / 1 mutation strategy, the mutation vector The generation method is as follows: ,in and These are two different individuals selected from the population. It is the best individual in the current population. It is the scaling factor.

[0013] In step S4 above, the method for determining the j-th dimension of the generated experimental individual is as follows: Among them, the crossover probability , Represents a random number uniformly distributed on [0,1]. It is a randomly selected dimension index.

[0014] In step S6 above, the surrogate model is adaptively selected as the Kriging model; when using the Kriging model, the Kriging surrogate model with constant mean is: ,in It is a zero-mean Gaussian random process; set For input points, For the corresponding objective function value, the Kriging model uses an anisotropic Gaussian correlation function, which has the following form: ,in This is the correlation hyperparameter.

[0015] Specifically, theoretical analysis and experiments show that the Kriging model has higher prediction accuracy in low-dimensional scenarios. Therefore, the Kriging model is preferred for low-dimensional subspaces. For higher-dimensional subspaces, the radial basis function model is used to balance computational efficiency and model performance.

[0016] The above represents hyperparameter learning as a bounded optimization problem: The problem is solved using a multi-starting-point direct search algorithm.

[0017] Specifically, the problem is solved using the multi-starting-point Hooke–Jeeves direct search algorithm.

[0018] The present invention also provides a bi-level optimization system for large-scale, expensive single-objective optimization problems, applying the bi-level optimization method for large-scale, expensive single-objective optimization problems as described above.

[0019] The present invention also provides a computer storage device that stores a plurality of instructions, which are adapted to be loaded by a processor and executed in steps S1-S7.

[0020] The beneficial effects of this invention are: This invention provides a two-layer optimization method and system for large-scale, expensive single-objective optimization problems. The method involves steps such as initialization, subproblem construction, surrogate model construction, subproblem optimization and evaluation, bottom-level population update, top-level surrogate model selection, and top-level optimization and global update. At the lower level, multiple candidate subspaces are constructed and evaluated, and key subspaces that significantly impact the global objective are automatically identified. These key subspaces are then optimized at the upper level, achieving efficient solutions to large-scale single-objective optimization problems within a finite expensive function evaluation budget. By introducing a surrogate model to approximate the expensive objective function and combining it with a two-layer optimization structure, the method organically integrates subspace selection and fine-tuning, effectively improving search efficiency, accelerating convergence speed, and enhancing the quality of the final solution, thus meeting the practical application needs of complex engineering optimization scenarios. Attached Figure Description

[0021] Figure 1 A flowchart illustrating the steps of the bi-layer optimization method for large-scale, expensive single-objective optimization problems provided by this invention. Figure 2 The flowchart illustrates the program execution logic of the bi-level optimization method for large-scale, expensive single-objective optimization problems provided by this invention. Detailed Implementation

[0022] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments. However, the scope of protection of this invention is not limited to the specific embodiments described below.

[0024] like Figures 1-2 As shown, this embodiment provides a two-layer optimization method for large-scale, expensive single-objective optimization problems. The two-layer optimization method includes: Step S1. Initialization: A set of diverse solutions is generated using Latin hypercube sampling, then evaluated using an expensive true function, and finally stored in the archive set. This collection of archives The training data and optimal solution repository for the surrogate model; and, for those containing Population of individuals Perform initialization processing to provide a search foundation for subsequent optimization; Step S2. Subproblem Construction: Generate subproblems sequentially. There are several subspaces, and the construction process of each subspace is as follows: in the interval... Randomly generate integers Determine the subspace dimension; then randomly select from the original problem decision variables. Each variable constitutes a subproblem The subproblem The decision space is represented as The corresponding temporary population is denoted as ; Step S3. Proxy Model Construction: From the Archives Random selection Extracting samples from sub-problems Based on the data of the corresponding variables, a radial basis surrogate model is trained. (i.e., the RBF model); the radial basis surrogate model It is a three-layer network structure, which includes an input layer, a hidden layer, and an output layer; the input points of the input layer... The quantity is of Population of decision variables, calculating sample points and sample points The distance between them is calculated as follows: , forming a matrix Use the cube kernel function to obtain the symmetric matrix of similarity between sample points. The calculation method is as follows: ; Then, use the augmented matrix A to construct a linear system to solve for the parameters. and A is composed of a symmetric matrix and polynomial part Composition; where P is the first-order polynomial extension matrix obtained by adding a constant term 1 to each row of the input point s, and the augmented matrix A is represented as: ; and by solving the system of linear equations To obtain parameters and Right-hand vector It is an input point target value And the zero vector of the tail part of the polynomial: , and They are The former The formula for constructing a surrogate model that can be used for direct prediction is: (The formula is missing from the original text.) ,in, For the polynomial part, yes The last element of the vector, For the new input point, This represents the corresponding prediction result.

[0025] Step S4. Sub-problem optimization and evaluation: In the first step... Subspace Within the subspace, a differential evolution algorithm is used to iteratively search for candidate solutions. In the population, new individuals are generated based on the DE / best / 1 mutation strategy and the binomial crossover operator; a binomial crossover is performed on the mutation vector and the target vector to generate experimental individuals; and a surrogate model is used for the generated experimental individuals. Predict its fitness value; after a preset number of iterative search rounds, from the subspace The solution with the best predicted fitness is obtained through screening. Its predicted fitness value This reflects the optimization potential of the subspace; by comparing the fitness of the optimal solutions to all subproblems, the current optimal subspace is determined; in the DE / best / 1 mutation strategy, the mutation vector... The generation method is as follows: ,in and These are two different individuals selected from the population. It is the best individual in the current population. It is a scaling factor. The j-th dimension of the generated experimental individual is determined as follows: Among them, the crossover probability , Represents a random number uniformly distributed on [0,1]. It is a randomly selected dimension index.

[0026] Step S5. Bottom Population Update Phase: Utilizing the current subproblem The optimization results update the original population The variable values ​​for the corresponding dimension; Step S6. Top-level surrogate model selection: For the selected optimal subspace, adaptively select a surrogate model based on its dimensional characteristics; the adaptively selected surrogate model is the Kriging model; when using the Kriging model, the Kriging surrogate model with constant mean is: ,in It is a zero-mean Gaussian random process; set For input points, For the corresponding objective function value, the Kriging model uses an anisotropic Gaussian correlation function, which has the following form: ,in Let these be the correlation hyperparameters. Hyperparameter learning can be represented as a bounded optimization problem: The problem is solved using the multi-starting-point Hooke–Jeeves direct search algorithm.

[0027] Step S7. Top-level optimization and global update stage: The particle swarm optimization algorithm with social learning mechanism is used to perform a fine search in the optimal subspace to obtain the local optimal solution in the subspace. Compare this solution with the optimal solution obtained from the underlying optimization. Combine and jointly update the global optimal solution The variable values ​​for the corresponding dimension.

[0028] This technical solution constructs a two-layer collaborative optimization mechanism during the overall optimization process, achieving hierarchical processing of subspace generation, screening, and fine-tuning. The lower layer constructs and evaluates multiple candidate subspaces in a high-dimensional decision space, while the upper layer performs in-depth searches within selected key subspaces, thereby improving the overall efficiency of large-scale single-objective optimization under a limited and expensive function evaluation budget. A surrogate model is used to predict the objective function values ​​of candidate solutions within different subspaces, and the subspaces are compared and ranked based on the prediction results, thus achieving quantitative evaluation and automatic screening of subspace importance. This mechanism effectively identifies key subspaces that significantly contribute to the improvement of the global objective function, providing clear and reliable search directions for subsequent optimization processes. Furthermore, by decoupling the subspace screening process from the subspace fine-tuning process in the algorithm structure, a collaborative optimization strategy is formed. Subspace screening focuses on quickly identifying potentially excellent search regions, while fine-tuning focuses on in-depth searches within selected key subspaces. This strategy avoids the efficiency degradation caused by the high coupling of the search process in traditional methods, improving the stability and convergence performance of the algorithm in high-dimensional search spaces.

[0029] Based on the disclosure and teachings of the foregoing specification, those skilled in the art can make changes and modifications to the above embodiments. Therefore, the present invention is not limited to the specific embodiments disclosed and described above, and some modifications and changes to the invention should also fall within the protection scope of the claims of the present invention. Furthermore, although some specific terms are used in this specification, these terms are only for convenience of explanation and do not constitute any limitation on the present invention.

Claims

1. A bi-level optimization method for large-scale, expensive single-objective optimization problems, characterized in that, The two-layer optimization method includes: Step S1. Initialization: A set of diverse solutions is generated using Latin hypercube sampling, then evaluated using an expensive true function, and finally stored in the archive set. This collection of archives The training data and optimal solution repository for the surrogate model; and, for those containing Population of individuals Perform initialization processing; Step S2. Subproblem Construction: Generate subproblems sequentially. There are several subspaces, and the construction process of each subspace is as follows: in the interval... Randomly generate integers Determine the subspace dimension; then randomly select from the original problem decision variables. Each variable constitutes a subproblem The subproblem The decision space is represented as The corresponding temporary population is denoted as ; Step S3. Proxy Model Construction: From the Archives Random selection Extracting samples from sub-problems Based on the data of the corresponding variables, a radial basis surrogate model is trained. ; Step S4. Sub-problem optimization and evaluation: In the first step... Subspace Within the subspace, a differential evolution algorithm is used to iteratively search for candidate solutions. In the population, new individuals are generated based on the DE / best / 1 mutation strategy and the binomial crossover operator; a binomial crossover is performed on the mutation vector and the target vector to generate experimental individuals; and a surrogate model is used for the generated experimental individuals. Predict its fitness value; after a preset number of iterative search rounds, from the subspace The solution with the best predicted fitness is obtained through screening. Determine the current optimal subspace; Step S5. Bottom Population Update Phase: Utilizing the current subproblem The optimization results update the original population The variable values ​​for the corresponding dimension; Step S6. Top-level proxy model selection: For the selected optimal subspace, adaptively select a proxy model based on its dimensional characteristics; Step S7. Top-level optimization and global update stage: The particle swarm optimization algorithm with social learning mechanism is used to perform a fine search in the optimal subspace to obtain the local optimal solution in the subspace. Compare this solution with the optimal solution obtained from the underlying optimization. Combine and jointly update the global optimal solution The variable values ​​for the corresponding dimension.

2. The two-layer optimization method according to claim 1, characterized in that, In step S3, the radial basis surrogate model It is a three-layer network structure, which includes an input layer, a hidden layer, and an output layer.

3. The two-layer optimization method according to claim 2, characterized in that, In step S3, the input point of the input layer The quantity is of Population of decision variables, calculating sample points and sample points The distance between them is calculated as follows: , forming a matrix ; Use the cube kernel function to obtain the symmetric matrix of similarity between sample points. The calculation method is as follows: ; Then, use the augmented matrix A to construct a linear system to solve for the parameters. and A is composed of a symmetric matrix and polynomial part Composition; where P is the first-order polynomial extension matrix obtained by adding a constant term to each row of the input point s, and the augmented matrix A is represented as: ; and by solving the system of linear equations To obtain parameters and Right-hand vector It is an input point target value And the zero vector of the tail part of the polynomial: , and They are The former The formula for constructing a surrogate model that can be used for direct prediction is: (The formula is missing from the original text.) ,in, For the polynomial part, yes The last element of the vector, For the new input point, This represents the corresponding prediction result.

4. The two-layer optimization method according to claim 1, characterized in that, In step S4, in the DE / best / 1 mutation strategy, the mutation vector The generation method is as follows: ,in and These are two different individuals selected from the population. It is the best individual in the current population. It is the scaling factor.

5. The two-layer optimization method according to claim 1, characterized in that, In step S4, the j-th dimension of the generated experimental individual is determined as follows: Among them, the crossover probability , Represents a random number uniformly distributed on [0,1]. It is a randomly selected dimension index.

6. The two-layer optimization method according to claim 1, characterized in that, In step S6, the surrogate model is adaptively selected as the Kriging model; when using the Kriging model, the Kriging surrogate model with constant mean is: ,in It is a zero-mean Gaussian random process; set For input points, For the corresponding objective function value, the Kriging model uses an anisotropic Gaussian correlation function, which has the following form: ,in This is the correlation hyperparameter.

7. The two-layer optimization method according to claim 6, characterized in that, The parameters of the Kriging model were estimated using the maximum likelihood estimation algorithm, and the regression coefficients and process variances are as follows: ,in ; Using Cholesky decomposition Construct the negative log-likelihood function: .

8. The two-layer optimization method according to claim 6, characterized in that, Representing hyperparameter learning as a bounded optimization problem: The problem is solved using a multi-starting-point direct search algorithm.

9. A bi-level optimization system for large-scale, expensive single-objective optimization problems, characterized in that, The bi-level optimization method for large-scale, expensive single-objective optimization problems is applied as described in any one of claims 1 to 8.

10. A computer storage device storing a plurality of instructions, characterized in that, The instructions are applicable to being loaded by a processor and executed as described in any one of claims 1-8.

Images