Optimization method, system and equipment for nonlinear programming model and medium
By converting linear equation systems into least squares problems under the framework of the internal point method and using long and short-term memory network algorithm to solve the problem of high resources, large computing power, and difficult to guarantee solution optimization in the existing technology, realizing The ability to batch processing optimization problems and internal point method hot start is achieved.
Patent Information
- Application Number
- CN202411093371.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-11-29
AI Technical Summary
When the existing learning optimization method deals with nonlinear planning problems with constraints, the resource and computing power requirements are high, and the changes in different optimization problems cannot be processed in batches. The loss function of traditional self-supervised learning is difficult to ensure the optimality and feasibility of the solution. , especially not suitable for hot start of the inner point method solver.
Under the framework of the inner point method, the linear equation system is converted into least squares problem, and the long and short-term memory network algorithm is used for solving. By combining the least squares algorithm and the long and short-term memory network, the variable values it iteratively updates until the inner point method is reached. The number of iterations ensures the optimality and feasibility of the output solution and is suitable for the hot start of the inner point method.
Implement the optimization problem samples that batch process all parameter changes within the internal point method framework, avoid the representation of constraint optimization problems, embed the update sequence of unconstrained optimization problems, reduce the need for hyperparameter adjustment, and ensure the optimality of understanding and Feasibility, suitable for hot start of the inner point method.
Smart Images

Figure CN119047305A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data processing, and in particular to an optimization method, system, device and medium for a nonlinear programming model. Background Art
[0002] Constrained nonlinear programming problems are a type of complex mathematical optimization problems characterized by nonlinear characteristics of the objective function, constraints, or both. The nonlinear relationships involved in such problems may originate from the inherent laws of physical processes, economic behaviors, or other complex systems. Common nonlinear programming problems include quadratic programs (QPs), quadratically constrained quadratic programs (QCQPs), semi-positive definite programming, etc. These planning problems can be further divided into convex or non-convex problems according to the properties of the objective function and constraints. Existing nonlinear programming problems are widely used in many fields such as power system planning and optimization, intelligent robot control, and wireless communication network resource allocation. They can better describe the complex dynamic processes in the real world and provide powerful mathematical modeling and solution tools for scientific research and engineering practice.
[0003] At present, learning optimization methods have been widely used in various optimization problems, involving unconstrained optimization, linear programming, and combinatorial optimization. The learning optimization methods for solving constrained nonlinear programming problems can be broadly divided into two main directions: directly inputting the optimization problem to obtain the solution to the problem, or aligning the learning optimization method with the traditional algorithm. Current research has adopted the former direct learning method, which trains the mapping relationship from the optimization input to the final solution through a supervised learning scheme. However, this method requires a large number of optimal solutions as training samples, and has high requirements for resources and computing power.
[0004] The mapping relationship from the optimization input to the final solution is trained through a supervised learning scheme. However, this method requires a large number of (near) optimal solutions as training samples, which has high requirements on resources and computing power.
[0005] In summary, the existing learning optimization methods still have the following problems: the existing optimization methods can only process specific optimization problems in batches and do not have the ability to represent general NLP problems. Therefore, they cannot batch process problems where the parameter matrices or vectors of multiple problems change between samples, such as only the vector on the right side of the equation changes between different optimization problems; the current optimization methods are not suitable for hot starting the interior point solver; the loss function penalty function based on traditional self-supervised learning has a penalty factor that needs to be manually adjusted, and it is difficult to ensure the optimality and feasibility of the solution. Summary of the invention
[0006] In order to address the deficiencies in the prior art, the present invention provides an optimization method, system, device and medium for a nonlinear programming model, which batch processes optimization problem samples with all parameter changes, and can naturally ensure the optimality and feasibility of the output solution at the same time within the framework of the interior point method.
[0007] The objective of the present invention is achieved by the following technical solutions:
[0008] An optimization method for a nonlinear programming model, comprising:
[0009] According to the initial primal variables and dual variables, the constructed nonlinear programming model is initially solved using the interior point method to obtain the corresponding linear equations.
[0010] The linear equations are input into the constructed optimization model, the initial primitive variables and the dual variables are updated, and approximate values are obtained, the optimization model is obtained based on the least squares algorithm and the long short-term memory network algorithm, the least squares algorithm is used to convert the linear equations into a least squares problem, and the long short-term memory network algorithm is used to solve the least squares problem to obtain updated values of the initial primitive variables and the dual variables;
[0011] According to the optimization model, the updated values of the initial primitive variables and the dual variable values are iteratively calculated until the number of iterations of the interior point method is reached, and an approximate optimal solution of the nonlinear programming model is obtained, and the approximate optimal solution is used as the optimal solution of the nonlinear programming model. As a further improvement of the present invention, the constructed nonlinear programming model is solved using the interior point method to obtain the corresponding linear equation system, including: extracting all the equality conditions in the KKT conditions based on the interior point method, and all the equality conditions constitute a nonlinear programming model, and the nonlinear programming model is:
[0012]
[0013] sth(x)=0
[0014] x≥0
[0015] In the formula, f(x) is the objective function, f(x) satisfies the second-order differentiability and h(x) is an equality constraint, h(x) satisfies the second-order differentiability and As a further improvement of the present invention, a nonlinear programming model is formed based on all the equation conditions in the KKT conditions, and the nonlinear programming model is solved by Newton's method to obtain the residual and Jacobian matrix of the nonlinear programming model, and a linear equation system is obtained through the residual and Jacobian matrix. The KKT conditions include:
[0016]
[0017] diag(z)diag(x)e=μex,z≥0
[0018] Wherein, when F(x,λ,z)=0, it represents the nonlinear equation group in the perturbation KKT condition, and the nonlinear equation group is:
[0019]
[0020] Where λ and z represent the dual variables corresponding to h(x) and x respectively, diag() is a diagonal matrix, e is a vector with elements equal to 1, and μ is the perturbation parameter.
[0021] As a further improvement of the present invention, the least squares algorithm is used to convert the linear equations into a least squares problem, including the expression of the least squares algorithm:
[0022]
[0023] Where || || is the Euclidean norm, J k is the coefficient matrix of the linear equations, F k is the right-hand side vector of the linear equations, k is the number of iterations of the interior point method, and y is the updated value of the primal and dual variables.
[0024] As a further improvement of the present invention, the linear equations are input into a constructed optimization model, the initial primal variables and the dual variables are updated, and approximate values are obtained, including that the optimization model is a trained model, and the training process is: obtaining a data set of a nonlinear programming problem, a learning rate, a number of training times, an iteration number of an interior point method, and a total time step of a long short-term memory network algorithm;
[0025] Obtain trainable parameters based on the long short-term memory network algorithm;
[0026] After converting the original variable and the dual variable values into the least squares problem, the corresponding gradient value is obtained, and the gradient value and the original variable are input into the long short-term memory network algorithm to obtain the current approximate update value;
[0027] Iterative calculation is performed based on the number of iterations of the interior point method and the total time step of the long short-term memory network algorithm, the corresponding gradient value is updated, and finally an approximate optimal solution is output.
[0028] As a further improvement of the present invention, the loss function in the long short-term memory network algorithm is used to update the gradient value in the approximate solution and the least squares module, and the loss function is:
[0029]
[0030] In the formula, is the total number of nonlinear programming problems in the data set, K is the total number of interior point method iterations, T is the total time step of the long short-term memory network algorithm, and J k is the coefficient matrix of the linear equations, is the updated value of the original variable and the dual variable at the kth interior point method iteration and the tth time step, θ is the optimal trainable parameter, F k is the right-hand side vector of the linear equations during the k-th interior point method iteration.
[0031] As a further improvement of the present invention, after obtaining the approximate optimal solution, the method further includes inputting the approximate optimal solution into a solver for hot start to obtain an optimal solution for the nonlinear programming model.
[0032] In a second aspect, the present invention further provides an optimization system for a nonlinear programming model, comprising:
[0033] The data acquisition module is used to obtain the values of the original variables and the dual variables and determine the number of iterations of the interior point method;
[0034] The initial solution module uses the interior point method to perform the initial solution to the constructed nonlinear programming model according to the initial primal variables and dual variables to obtain the corresponding linear equations.
[0035] An optimization module, wherein the linear equations are input into a constructed optimization model, initial primitive variables and dual variables are updated, and approximate values are obtained. The optimization model is obtained based on a least squares algorithm and a long short-term memory network algorithm. The least squares algorithm is used to convert the linear equations into a least squares problem. The long short-term memory network algorithm is used to solve the least squares problem to obtain updated values of the initial primitive variables and dual variables.
[0036] The problem solving module iteratively calculates the updated values of the initial primal variables and the dual variable values according to the optimization model until the number of iterations of the interior point method is reached, thereby obtaining an approximate optimal solution of the nonlinear programming model, and taking the approximate optimal solution as the optimal solution of the nonlinear programming model.
[0037] In a third aspect, the present invention further provides an electronic device, comprising a processor and a memory, wherein the processor is used to execute a computer program stored in the memory to implement the above-mentioned optimization method for a nonlinear programming model.
[0038] In a fourth aspect, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores at least one instruction, and when the at least one instruction is executed by a processor, the optimization method for a nonlinear programming model as described above is implemented.
[0039] The beneficial effects of the present invention are as follows: an optimization method for a nonlinear programming model of the present invention is based on an existing optimization algorithm, by converting a linear equation group solved by an interior point method into an unconstrained least squares related problem under the framework of an interior point method, and embedding it into a series of update values and corresponding gradient values and then inputting it into a long short-term memory network, thereby avoiding the representation of the constrained optimization problem, or embedding the representation of the constrained optimization problem into the update sequence of the unconstrained optimization problem, so that our algorithm can batch process optimization problem samples in which all parameters have changed.
[0040] Furthermore, through the loss function based on the least square problem transformed from the linear system, the loss function has no additional hyperparameters that need to be adjusted, and the feasibility and optimality of the solution can be guaranteed at the same time through the interior point method. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0042] Figure 1 is a schematic flow chart of an optimization method for a nonlinear programming model in an embodiment of the present invention;
[0043] Figure 2 is a schematic diagram of the structure of a long short-term memory network algorithm in an embodiment of the present invention;
[0044] Figure 3 Schematic diagram of the training process of the long short-term memory network algorithm in an embodiment of the present invention;
[0045] Figure 4 It is a schematic diagram of the structure of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION
[0046] In order to make the purpose and technical solution of the present invention clearer and easier to understand, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. The specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0047] The concept of the present invention is to provide an optimization method, system, device and medium for a nonlinear programming model, the method mainly comprising:
[0048] Get the values of the original and dual variables and determine the number of iterations of the interior point method;
[0049] According to the initial primal variables and dual variables, the constructed nonlinear programming model is initially solved using the interior point method to obtain the corresponding linear equations.
[0050] The linear equations are input into the constructed optimization model, the initial primitive variables and the dual variables are updated, and approximate values are obtained, the optimization model is obtained based on the least squares algorithm and the long short-term memory network algorithm, the least squares algorithm is used to convert the linear equations into a least squares problem, and the long short-term memory network algorithm is used to solve the least squares problem to obtain updated values of the initial primitive variables and the dual variables;
[0051] The updated values of the initial primal variables and the dual variables are iteratively calculated according to the optimization model until the number of iterations of the interior point method is reached, and an approximate optimal solution of the nonlinear programming model is obtained, and the approximate optimal solution is used as the optimal solution of the nonlinear programming model.
[0052] Terminology explanation:
[0053] Nonlinear Programming (NLP): Nonlinear programming problem is a type of optimization problem that contains nonlinear functions in the objective function or constraints.
[0054] Learning to Optimize (L2O): Learning optimization methods refer to a class of methods that combine machine learning methods with traditional methods or directly use machine learning methods to solve various optimization problems.
[0055] Long Short-Term Memory Network (LSTM): It is a recurrent neural network composed of gating mechanisms such as forget gate, input gate, and output gate. It is mainly used to process sequence data and can effectively capture and retain dependency information in long sequence data.
[0056] The Interior Point Method (IPM) is a method often used to solve linear and nonlinear programming. It uses Newton's method and line search to iteratively solve a perturbed KKT condition with a parameter μ tending to 0, and finally reaches the KKT point of the original problem. The KKT condition includes gradient conditions, constraint conditions, and mutual relaxation conditions.
[0057] IPOPT: An open source nonlinear optimization solver based on the interior point method developed jointly by Carnegie Mellon University and the University of Illinois at Urbana-Champaign.
[0058] Least squares problem: The least squares problem is a type of problem of the form min||Ax–b||2 2 Convex optimization problem.
[0059] End to End: End to End is a training and solving method for deep learning models. It refers to inputting the original features of the data without feature engineering and then outputting the results required for the task. In the scenario of learning optimization, it refers to mapping the optimization problem into features input into the network and outputting the solution to the optimization problem.
[0060] Warm Start: Warm start means that when using a solver to solve an optimization problem, an initial solution is first passed to the solver and then the following solution process is continued, aiming to speed up the solver's solution process.
[0061] The technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings and specific embodiments, wherein the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments.
[0062] Example 1
[0063] like Figure 1 As shown, an optimization method for a nonlinear programming model in this embodiment first gives an initial primitive variable and dual variable value x0, λ0, z0, and uses the one-step Newton method to solve all nonlinear equations in the KKT condition to obtain a linear system whose solution variables are the update amounts of the primitive variables and the dual variables, and then converts it into a least squares problem. The representation method of the unconstrained optimization problem allows our algorithm to handle general nonlinear programming problems; then an initial y0 is given, the update value is concatenated with the gradient value of the least squares problem and input into the LSTM, and the update values required for the primitive variables and the dual variables are output; in this way, it is repeatedly iterated in the framework of the interior point method until a maximum number of interior point method iterations is set, and a high-quality approximate optimal solution to the original problem is output; finally, it is handed over to the interior point method solver for hot start to obtain the optimal solution to the original problem.
[0064] The main implementation steps of this method include the following.
[0065] First, obtain the values of the primal and dual variables and determine the number of iterations of the interior point method.
[0066] According to the initial primal variables and dual variables, the constructed nonlinear programming model is initially solved using the interior point method to obtain the corresponding linear equations.
[0067] Among them, the nonlinear programming model in this embodiment is:
[0068]
[0069] sth(x)=0
[0070] x≥0
[0071] In the formula, f(x) is the objective function, f(x) satisfies the second-order differentiability and is an equality constraint, h(x) satisfies the second-order differentiability and is a real number. Secondly, the linear equations are input into the constructed optimization model, the initial primitive variables and dual variables are updated, and approximate values are obtained. The optimization model is obtained based on the least squares algorithm and the long short-term memory network algorithm. The least squares algorithm is used to convert the linear equations into a least squares problem, and the long short-term memory network algorithm is used to solve the least squares problem to obtain the updated values of the initial primitive variables and dual variables.
[0072] Based on all the equation conditions, a nonlinear programming model is constructed, and the Newton method is used to solve the nonlinear programming model to obtain the residual and Jacobian matrix of the nonlinear programming model, and a linear equation system is obtained through the residual and Jacobian matrix. This embodiment uses the primal dual interior point method, which involves iteratively solving the perturbed KKT condition under a parameter sequence that is asymptotically close to zero:
[0073]
[0074] diag(z)diag(x)e=μex,z≥0
[0075] Where λ and z represent the dual variables corresponding to h(x) and x, respectively, diag() is a diagonal matrix, e is a vector with elements of 1, μ is a perturbation parameter, and F(x,λ,z)=0 represents the nonlinear equation system in the perturbation KKT condition, which is solved using Newton's method:
[0076]
[0077] The interior point method starts from an initial feasible solution (x0, λ0, z0), where x0, z0>0. In the kth iteration, we k ,λ k , z k ) to construct a linear equation system. The calculation expression of the disturbance parameter μ is:
[0078] μ=σ[(z k ) T x k ] / n
[0079] Among them, σ∈(0,1) is the first constant. Along the direction (Δx k , Δλ k , Δz k) to ensure that boundary conditions are met and sufficient progress is made in terms of objective value improvement or constraint violation reduction. This process is iterated until convergence criteria are met, such as optimality and feasibility within a specified tolerance.
[0080] Since the interior point method has a small number of iterations, it does not always guarantee that it can be solved efficiently, because sometimes the complexity of the interior point method is relatively high when solving linear systems. Therefore, this embodiment uses an optimization model to optimize the process. That is, the linear equations are input into the constructed optimization model, the initial primitive variables and dual variable values are updated, and approximate values are obtained. The optimization model includes a least squares module and a long short-term memory network algorithm connected in sequence.
[0081] Finally, the update values of the initial primal variables and the dual variables are iteratively calculated according to the optimization model until the number of iterations of the interior point method is reached, and the approximate optimal solution of the nonlinear programming model is obtained, and the approximate optimal solution is used as the optimal solution of the nonlinear programming model. After obtaining the approximate optimal solution, it also includes inputting the approximate optimal solution into the solver for hot start to obtain the optimal solution of the nonlinear programming model. Among them, the data information of the optimization problem is embedded into the matrix and the residual by using the long short-term memory network algorithm. After the linear equations are converted by the least squares algorithm, the least squares gradient value is formed with the set initial approximate value. The data information of the optimization problem is embedded into the least squares gradient value, and the long short-term memory network algorithm is used to process the sequence data formed by the approximate value and its gradient value. In other words, y is obtained by solving the linear system composed of the Jacobian matrix and the residual of the nonlinear equation, where y contains the update amount of the primal variable and the dual variable. We convert the linear system into a least squares problem and use LSTM to iteratively process the sequence data Where t represents the tth time step of the LSTM. The second part of the sequence data is the gradient of the input, which is the least squares problem about gradient.
[0082] Specifically, the computational complexity of solving Newton's equations using decomposition or iteration may reach O(n 3 ) floating point operations. This motivates us to avoid computing exact solutions to linear systems of equations and instead focus on their approximations. The least squares algorithm consists of:
[0083]
[0084] In the formula, |||| is the Euclidean norm, J k is the coefficient matrix of the linear equations, F k is the right-hand side vector of the linear equations, k is the number of iterations of the current interior point method, and y is the updated value of the primal and dual variables.
[0085] The optimal solution of the least squares algorithm is equal to the interior point method linear equation system: [(Δx k ) T , (Δλ k ) T , (Δz k ) T ] T . Approximately solve the system of equations.
[0086] After being solved by the least squares algorithm, the learning optimization problem is effectively solved through the characteristics of cyclic calculations in the long short-term memory network architecture.
[0087] The LSTM network consists of T units parameterized with the same learnable parameters. Each unit can be regarded as an iteration of the traditional iterative method, such as Figure 2 This embodiment defines:
[0088]
[0089] The tth unit takes the approximate value y obtained last time t-1 and gradient As input, output the current approximation y t ,Right now:
[0090] y t :=SLTM θ ([y t-1 ,(J k ) T (J k y t-1 +F k )])
[0091] The Tth unit outputs y T As an approximate solution to the least squares problem. At the same time, using a coordinate shared parameter LSTM, not only the parameters are shared between different LSTM units, but also the parameters are shared on all coordinates. The specific expression of LSTM in this embodiment is:
[0092] F t+1 =σ(Z t W f +H t U f +b f )
[0093] I t+1 =σ(Z t W i +H t U i +b i )
[0094] Ot+1 =σ(Z t W o +H t U o +b o )
[0095]
[0096] H t+1 =O t+1 tanh(C t+1 )
[0097] G t+1 =H t+1 W h +b h
[0098] y t+1 =y t +G t+1
[0099] in, is the input at each moment, H t is the hidden state of LSTM, C t is the memory state of LSTM, W, U, and b are the training parameter matrices and biases. We perform another linear transformation on the newly obtained hidden state to obtain G t+1 Used to update the variable y. The entire LSTM process is like imitating a first-order algorithm for solving the least squares problem, where the variable G can correspond to the update amount in the first-order algorithm. The information of an optimization problem is first embedded into the Jacobian matrix and residual of the Newton method; then we express the linear system as an unconstrained least squares problem. Given an initial value y, we then embed the information of the optimization problem into the gradient value of y and its corresponding least squares problem; finally, use LSTM to process the sequence data formed by y and its gradient value. In this way, we can enable our algorithm to batch process optimization problems with multiple parameter changes.
[0100] Since the LSTM network solution system simulates a first-order iterative method, the value of the condition number will seriously affect the performance of the LSTM network. To solve this problem, we use a simple diagonal preprocessing technique to scale the Heather matrix using the Ruiz scaling method to reduce its condition number.
[0101] The optimization model is a trained model. The training process of the optimization model is as follows: Figure 3 The training process is as follows: obtaining the data set of the nonlinear programming problem, the learning rate, the number of training times, the number of iterations of the interior point method, and the total time step of the long short-term memory network algorithm;
[0102] Obtain trainable parameters based on the long short-term memory network algorithm;
[0103] After converting the original variable and the dual variable values into the least squares problem, the corresponding gradient value is obtained, and the gradient value and the original variable are input into the long short-term memory network algorithm to obtain the approximate update value; iterative calculation is performed based on the number of iterations of the interior point method and the total time step of the long short-term memory network algorithm, the corresponding gradient value is updated, and the approximate optimal solution is finally output. The y output by LSTM is k is the update amount of x,λ,z for the kth iteration of the interior point method. The step size is adjusted by the line search of the interior point method itself, combined with the update amount y k Update x,λ,z together.
[0104] In order to adapt the training of this model, we designed a new self-supervised loss function based on the least squares problem. Compared with the traditional self-supervised loss function, there are no hyperparameters that need to be manually adjusted.
[0105] The loss function in the long short-term memory network algorithm is used to update the gradient value in the approximate solution and the least squares algorithm. The loss function is:
[0106]
[0107] In the formula, is the total number of nonlinear programming problems in the data set, K is the total number of interior point method iterations, T is the total time step of the long short-term memory network algorithm, and J k is the coefficient matrix of the linear equations, is the updated value of the original variable and the dual variable at the kth interior point method iteration and the tth time step, θ is the optimal trainable parameter, F k is the right-hand side vector of the linear equations during the k-th interior point method iteration.
[0108] Obviously, this loss function does not require the optimal solution or approximate optimal solution of the original problem, so our algorithm also belongs to the category of self-supervised training. Compared with the traditional self-supervised loss function based on penalty function, the loss function based on least squares problem does not have hyperparameters like penalty factor that need to be adjusted, which is a significant advantage.
[0109] This embodiment converts the linear system solved by the interior point method into an unconstrained least squares problem within the framework of the interior point method, and embeds it into a series of update values and corresponding gradient values and then inputs them into the LSTM, thereby avoiding the representation of the constrained optimization problem, or embedding the representation of the constrained optimization problem into the update sequence of the unconstrained optimization problem, so that our algorithm can batch process optimization problem samples in which all parameters have changed.
[0110] At the same time, the loss function of the least squares problem transformed from a linear system has no additional hyperparameters that need to be adjusted, and the feasibility and optimality of the solution can be guaranteed at the same time through the interior point method.
[0111] In addition, the learning optimization algorithm based on the interior point method can output the original dual solution and the initial μ value required by the interior point method, so it is more suitable for the hot start of the interior point method.
[0112] In summary, this embodiment can batch process optimization problem samples with all parameter changes, and can naturally ensure the optimality and feasibility of the output solution within the framework of the interior point method. Secondly, the new loss function does not have parameters that need to be manually adjusted and is easy to train, so it has obvious advantages over existing algorithms.
[0113] Example 2
[0114] This embodiment provides an optimization system for a nonlinear programming model, including:
[0115] The data acquisition module is used to obtain the values of the original variables and the dual variables and determine the number of iterations of the interior point method;
[0116] The initial solution module uses the interior point method to perform the initial solution to the constructed nonlinear programming model according to the initial primal variables and dual variables to obtain the corresponding linear equations.
[0117] An optimization module, wherein the linear equations are input into a constructed optimization model, initial primitive variables and dual variables are updated, and approximate values are obtained. The optimization model is obtained based on a least squares algorithm and a long short-term memory network algorithm. The least squares algorithm is used to convert the linear equations into a least squares problem. The long short-term memory network algorithm is used to solve the least squares problem to obtain updated values of the initial primitive variables and dual variables.
[0118] The problem solving module iteratively calculates the updated values of the initial primitive variables and the dual variables according to the optimization model until the number of iterations of the interior point method is reached, and obtains an approximate optimal solution of the nonlinear programming model, and uses the approximate optimal solution as the optimal solution of the nonlinear programming model. The operation steps corresponding to each module have been specifically described in the optimization method for the nonlinear programming model in Implementation 1, and will not be repeated here.
[0119] Example 3
[0120] like Figure 4The embodiment shown provides an electronic device for implementing the optimization method for nonlinear programming model in the above-mentioned embodiment 1. The electronic device 100 includes at least one processor 102, a memory 101, a computer program 103 stored in the memory 101 and executable on at least one processor 102, and at least one communication bus 104. The memory 101 can be used to store the computer program 103. The processor 102 implements the steps of the optimization method for nonlinear programming model in embodiment 1 by running or executing the computer program stored in the memory 101 and calling the data stored in the memory 101. The memory 101 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data (such as audio data) created according to the use of the electronic device 100, etc. In addition, the memory 101 may include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices.
[0121] At least one processor 102 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 102 may be a microprocessor or any conventional processor, etc. The processor 102 is the control center of the electronic device 100, and uses various interfaces and lines to connect various parts of the entire electronic device 100.
[0122] The memory 101 in the electronic device 100 stores a plurality of instructions to implement an optimization method for a nonlinear programming model, and the processor 102 can execute the plurality of instructions to implement:
[0123] Get the values of the original and dual variables and determine the number of iterations of the interior point method;
[0124] According to the initial primal variables and dual variables, the constructed nonlinear programming model is initially solved using the interior point method to obtain the corresponding linear equations.
[0125] The linear equations are input into the constructed optimization model, the initial primitive variables and the dual variables are updated, and approximate values are obtained, the optimization model is obtained based on the least squares algorithm and the long short-term memory network algorithm, the least squares algorithm is used to convert the linear equations into a least squares problem, and the long short-term memory network algorithm is used to solve the least squares problem to obtain updated values of the initial primitive variables and the dual variables;
[0126] The updated values of the initial primal variables and the dual variables are iteratively calculated according to the optimization model until the number of iterations of the interior point method is reached, and an approximate optimal solution of the nonlinear programming model is obtained, and the approximate optimal solution is used as the optimal solution of the nonlinear programming model.
[0127] Example 4
[0128] If the module / unit integrated in the electronic device 100 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. Computer-readable media may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory and read-only memory (ROM, Read-Only Memory).
[0129] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0130] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0131] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.
Claims
1. An optimization method for a nonlinear programming model, characterized in that: include: Get the values of the original and dual variables and determine the number of iterations of the interior point method; According to the initial primal variables and dual variables, the constructed nonlinear programming model is initially solved using the interior point method to obtain the corresponding linear equations. The linear equations are input into the constructed optimization model, the initial primitive variables and the dual variables are updated, and approximate values are obtained, the optimization model is obtained based on the least squares algorithm and the long short-term memory network algorithm, the least squares algorithm is used to convert the linear equations into a least squares problem, and the long short-term memory network algorithm is used to solve the least squares problem to obtain updated values of the initial primitive variables and the dual variables; The updated values of the initial primal variables and the dual variables are iteratively calculated according to the optimization model until the number of iterations of the interior point method is reached, and an approximate optimal solution of the nonlinear programming model is obtained, and the approximate optimal solution is used as the optimal solution of the nonlinear programming model.
2. The optimization method for a nonlinear programming model according to claim 1, characterized in that: The method uses the interior point method to solve the constructed nonlinear programming model to obtain the corresponding linear equation system, including: extracting all the equation conditions in the KKT conditions based on the interior point method, and all the equation conditions constitute the nonlinear programming model, and the nonlinear programming model is: sth(x)=0 x≥0 In the formula, f(x) is the objective function, f(x) satisfies the second-order differentiability and h(x) is an equality constraint, h(x) satisfies the second-order differentiability and is a real number.
3. The optimization method for a nonlinear programming model according to claim 2, characterized in that: A nonlinear programming model is constructed based on all the equation conditions in the KKT condition, and the nonlinear programming model is solved by Newton's method to obtain the residual and Jacobian matrix of the nonlinear programming model, and a linear equation system is obtained through the residual and Jacobian matrix. The KKT condition includes: diag(z)diag(x)e=μex,z≥0 Wherein, when F(x,λ,z)=0, it represents the nonlinear equation group in the perturbation KKT condition, and the nonlinear equation group is: Where λ and z represent the dual variables corresponding to h(x) and x respectively, diag() is a diagonal matrix, e is a vector with elements equal to 1, and μ is the perturbation parameter.
4. The optimization method for a nonlinear programming model according to claim 3, characterized in that: The least squares algorithm is used to convert a linear equation system into a least squares problem, and the expression of the least squares algorithm is: In the formula, |||| is the Euclidean norm, J k is the coefficient matrix of the linear equations, F k is the right-hand side vector of the linear equations, k is the number of iterations of the current interior point method, and y is the updated value of the primal and dual variables.
5. The optimization method for a nonlinear programming model according to claim 1, characterized in that: The linear equations are input into the constructed optimization model, the initial primitive variables and dual variables are updated, and approximate values are obtained. Including, the constructed optimization model is a trained model, and the training process is: obtaining a data set of a nonlinear programming problem, a learning rate, a number of training times, a number of iterations of an interior point method, and a total time step of a long short-term memory network algorithm; The training parameters are obtained based on the long short-term memory network algorithm; After the linear equations are converted into a least squares problem, they are input into the long short-term memory network algorithm to obtain the updated values of the initial primal variables and dual variables. Iterative calculation is performed based on the number of iterations of the interior point method and the total time step of the long short-term memory network algorithm, and the initial primal variables and dual variable values are continuously updated until the maximum number of iterations of the interior point method is reached, and finally an approximate optimal solution is output.
6. The optimization method for a nonlinear programming model according to claim 5, characterized in that: The loss function in the long short-term memory network algorithm is obtained based on the least squares algorithm, and the loss function is: Where M is the total number of nonlinear programming problems in the data set, K is the total number of interior point method iterations, T is the total time step of the long short-term memory network algorithm, and J is the total number of interior point method iterations. k is the coefficient matrix of the linear equations, is the updated value of the original variable and the dual variable at the kth interior point method iteration and the tth time step, θ is the optimal trainable parameter, F k is the right-hand side vector of the linear equations during the k-th interior point method iteration.
7. The optimization method for a nonlinear programming model according to claim 1, characterized in that: After obtaining the approximate optimal solution, the method further includes inputting the approximate optimal solution into a solver for hot start to obtain the optimal solution of the nonlinear programming model.
8. An optimization system for a nonlinear programming model, characterized in that: include: The data acquisition module is used to obtain the values of the original variables and the dual variables and determine the number of iterations of the interior point method; The initial solution module uses the interior point method to perform the initial solution to the constructed nonlinear programming model according to the initial primal variables and dual variables to obtain the corresponding linear equations. An optimization module, wherein the linear equations are input into a constructed optimization model, initial primitive variables and dual variables are updated, and approximate values are obtained. The optimization model is obtained based on a least squares algorithm and a long short-term memory network algorithm. The least squares algorithm is used to convert the linear equations into a least squares problem. The long short-term memory network algorithm is used to solve the least squares problem to obtain updated values of the initial primitive variables and dual variables. The problem solving module iteratively calculates the updated values of the initial primal variables and the dual variable values according to the optimization model until the number of iterations of the interior point method is reached, thereby obtaining an approximate optimal solution of the nonlinear programming model, and taking the approximate optimal solution as the optimal solution of the nonlinear programming model.
9. An electronic device, characterized in that: The system comprises a processor and a memory, wherein the processor is used to execute a computer program stored in the memory to implement the optimization method for a nonlinear programming model as claimed in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, and when the at least one instruction is executed by a processor, the optimization method for a nonlinear programming model according to any one of claims 1 to 7 is implemented.
Citation Information
Cited By
Signal analysis method, system, equipment and medium based on quadratic constraint quadratic programming problem solution prediction
CN120031171A
Heterogeneous acceleration method and device for foot type robot control and storage medium
CN120170732A
