Application method of physical information neural network for optimal control problem

Abstract

The invention discloses a use method of a physical information neural network for an optimal control problem, and belongs to the field of optimal control problems. The method comprises the following steps: adding physical constraints into a loss function of a physical information neural network, and dynamically adjusting weights according to loss changes in a training process through a self-adaptive weight mechanism, so that the network pays more attention to an area with relatively large loss, thereby improving the solving precision of a model for complex problems. According to the method, under the condition of not needing any data, by fusing constraints of a physical equation and using a self-adaptive weight mechanism, the optimal control law is more accurately approached, a traditional method for refining a grid is replaced, and the problem that in the face of two-dimensional, three-dimensional or high-dimensional problems, the optimal control law is more accurately approached by the traditional method is solved. The calculation amount is large due to the fact that the grids need to be refined continuously.

Description

Technical Field

[0001] The present invention belongs to the field of optimal control problems, and specifically provides a method for solving optimal control problems based on a physical information neural network with an adaptive weight mechanism. Background Art

[0002] Optimal control problems frequently arise in systems engineering. The goal is to determine a control strategy that optimizes system performance while satisfying the dynamic system and constraints. Optimal control theory seeks to find control laws for a given system that achieve a specific optimization criterion. Optimal control, a core area of ​​modern control theory, aims to optimize the performance of control systems. Drawing from numerous practical problems, it investigates how to identify the optimal solution from a range of allowable control options, ensuring optimal system performance as the system transitions from an initial state to a target state.

[0003] Among traditional approaches to solving optimal control problems, pseudospectral methods are a widely used numerical method. By parameterizing the state and control variables using global interpolating polynomials, pseudospectral methods transform the optimal control problem into a nonlinear programming problem (NLP), enabling efficient solution.

[0004] Despite their widespread application, pseudospectral methods also face numerous challenges. They are limited in their ability to handle discontinuities or high-order nonsmoothness in state or control variables, necessitating the use of adaptive meshing techniques to increase grid points in these areas to improve accuracy. Consequently, computational costs rise sharply with dimensionality, particularly in two-dimensional or higher-dimensional problems, and they face challenges such as error accumulation, complex constraints, and algorithmic stability.

[0005] To overcome these limitations, the combination of neural networks and optimal control has become a research hotspot in recent years. Using neural networks in optimal control offers several advantages, including the ability to handle high-dimensional state spaces, nonlinear dynamics, and complex constraints, which are often challenging for traditional methods. Neural networks are increasingly being used in optimal control problems to approximate control laws or value functions, providing a flexible and efficient framework for solving complex control problems.

[0006] With the rapid development of deep learning technology, artificial intelligence has achieved remarkable success in many fields. However, in the field of engineering design, obtaining labeled data is often costly and time-consuming, which greatly limits the widespread application of these advanced technologies. To solve this problem, the paper "Raissi M, Perdikaris P, Karniadakis G E. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations [J]. Journal of Computational Physics, 2019, 378: 686-707." first proposed the "Physics-Informed Neural Networks" (PINN). By leveraging the flexibility and scalability of neural networks, PINN can effectively solve physical problems even with little or no data.

[0007] In addition, physical information neural networks also have great advantages in solving two-dimensional or high-dimensional problems. The contribution of this invention is to solve two-dimensional and three-dimensional optimal control problems through physical information neural networks, resolving the challenges of traditional numerical methods such as extremely high computational costs due to grid refinement and the addition of interpolation points. The document "Wang Xinwei. Symplectic pseudospectral method for nonlinear optimal control problems and its application [D]. Dalian University of Technology, 2019" mentions that pseudospectral methods also face challenges in computational complexity and solution efficiency when dealing with high-dimensional systems or complex nonlinear systems. Although grid adaptive technology can capture discontinuities, in areas where the control input or state variables change drastically, the continuous refinement of the grid will lead to a significant increase in the amount of computation. Compared with traditional numerical methods, physical information neural networks can flexibly handle complex geometric and high-dimensional problems, and combine physical models with a small amount of observation data to obtain more accurate approximate solutions. When PINN solves the optimal control problem, it first constructs a neural network with the problem's independent variables as input and the dependent variables as output. It then defines a comprehensive loss function that includes a physical information loss term (in which the equation corresponding to the optimal control problem is embedded), a data matching loss term (if there is observation data), an initial condition loss term, and a boundary condition loss term. An optimization algorithm is then used to iteratively update the network parameters to minimize the loss function, thereby obtaining an approximate solution that satisfies the optimal control problem.

[0008] Furthermore, in physical-information neural networks, boundary conditions, interface conditions, and the partial differential equation (PDE) itself are incorporated into the loss function, forcing the network to meet these conditions as much as possible during training. This approach has the advantage of being simple to implement and unifying all conditions within a single loss function for optimization. However, the effectiveness of combining PDEs with boundary and interface conditions during training depends largely on the choice of weights in the loss function. The paper "Lai MC, Song Y, Yuan X, et al. The Hard-Constraint PINNs for Interface Optimal Control Problems [J]. arXiv preprint arXiv: 2308.06709, 2023" states that "there are no established rules or principles for systematically determining weights, and manually setting them through trial and error is extremely challenging and time-consuming." Therefore, to address the "deficiencies" in weight selection in physics-informed neural networks, the paper "McClenny L D, Braga-Neto UM. Self-adaptive physics-informed neural networks [J]. Journal of Computational Physics, 2023, 474: 111722" proposes using an adaptive weight mechanism combined with physics-informed neural networks to solve various problems. The basic idea of ​​the weight adaptation mechanism is to increase the weight as the corresponding loss increases. This is achieved by training the network to simultaneously minimize the loss and maximize the weight.

[0009] This paper proposes a physically-informed neural network (PINN) based on an adaptive weighting mechanism to solve two- and three-dimensional optimal control problems, and demonstrates its feasibility by comparing it with analytical solutions. PINN achieves robust generalization and excellent interpretability by integrating physical constraints and prior knowledge into a neural network architecture without using data. Summary of the Invention

[0010] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is as follows:

[0011] A method for solving optimal control problems using a physical information neural network based on an adaptive weight mechanism comprises the following steps:

[0012] Data and formula acquisition: Collect relevant information of the optimal control problem, including boundary conditions, initial conditions, physical formulas or mathematical expressions, and functional formulas;

[0013] Neural network construction: Use PyCharm to build the neural network structure and set the input layer, hidden layer, and output layer. Determine the number of input elements and output elements based on the specific requirements of the problem; and reasonably set the number of hidden layers and neurons based on the difficulty of the problem.

[0014] Parameter initialization and activation function setting: Initialize the adaptive weight to 1 and select the hyperbolic tangent function (tanh) as the activation function;

[0015] Loss function definition: Based on the obtained boundary conditions, initial conditions, physical formulas or mathematical expressions, and functional formulas, boundary loss, initial loss, PDE loss, and functional loss are constructed respectively;

[0016] Network Training and Results Comparison: Train the physical information neural network by performing forward propagation, calculating the loss values ​​for each component of the loss function, and then performing backpropagation to update the parameters. After training is complete, plot the graph and compare the neural network's predicted solution with the exact solution to evaluate the model's performance.

[0017] The beneficial effects of the present invention are as follows: the present invention provides a method for solving optimal control problems based on a physical information neural network with an adaptive weight mechanism, and proposes a physical information neural network framework that does not use any data but only uses physical formulas, boundary conditions, initial conditions and functional constraints. By incorporating an adaptive weight mechanism, it solves two-dimensional and three-dimensional optimal control problems that are difficult to solve with traditional numerical methods, thereby improving the generalization of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings used in the embodiments.

[0019] Figure 1 is a flowchart of the steps of the method for solving the optimal control problem using a physical information neural network based on an adaptive weight mechanism described in an embodiment;

[0020] Figure 2 2. The predicted result diagram (a) and relative error diagram (b) of the two-dimensional diffusion equation in the embodiment;

[0021] Figure 3 3D diffusion equations in the embodiment predict the two-dimensional cross-sectional view (a), three-dimensional view (b) and relative error view (c). DETAILED DESCRIPTION

[0022] To make the technical solution of the present invention clearer, the present invention will be clearly and completely explained below with reference to the accompanying drawings of the embodiments of the invention. The embodiments described only use a part of the embodiments, not all of the embodiments. The present invention is specifically implemented according to the following steps:

[0023] (1) Data and formula acquisition: Collect relevant information about the optimal control problem, including boundary conditions, initial conditions, physical formulas or mathematical expressions, and functional formulas. This process is described by formulas (1) to (4):

[0024] (1) Target functional:

[0025]

[0026] (2) Equation of state:

[0027]

[0028]

[0029] (3) The function given in the example:

[0030]

[0031] u0(x)=sin(πx),

[0032] (4) Exact solution:

[0033] u=sin(πx)cos(πγ),

[0034]

[0035] (2) Neural network construction: Use PyCharm to build the neural network structure and set the input layer, hidden layer, and output layer. Determine the number of input elements and output elements based on the specific requirements of the problem; and reasonably set the number of hidden layers and neurons based on the difficulty of the problem.

[0036] For the two-dimensional diffusion equation problem, the input layer is set to 2 (x, t), where x is a spatial variable describing the spatial distribution of the state variable, and t represents time. The output layer is set to 3 (u, z, q), where u is the state variable that satisfies the reaction-diffusion equation and reflects the dynamic behavior of the system. z is the adjoint state variable, obtained through the adjoint equation, describing the system's sensitivity to the objective function. q is the reaction coefficient, representing the system's reaction strength and influencing the dynamic behavior of the state variable. The number of hidden layers is set to 4, and the number of neurons is set to 64.

[0037] (3) Parameter initialization and activation function setting: Initialize the adaptive weight to 1 and select the hyperbolic tangent function (tanh) as the activation function;

[0038] In the adaptive weighting mechanism of PINN, the strategy is to minimize loss and maximize weight. The principle is as follows: During training, the adaptive weights are updated using gradient ascent by calculating the gradient of the loss function with respect to the network parameters and the adaptive weights. This means that during backpropagation, the gradient of the adaptive weights is negated, and the parameters are then updated using the optimizer. When the loss of a training sample or region is large, the corresponding gradient is also large. Updating the weights along this gradient direction will increase the weight, causing the network to pay more attention to these difficult-to-learn samples or regions in subsequent training.

[0039] The figure shows the mathematical method for dynamic adjustment of the physical information neural network with adaptive weight mechanism:

[0040]

[0041] Among them, ηk represents the learning rate of the network weight at the kth step, which controls the update step size of the weight; Represents the gradient of the loss function L with respect to the network weight w, indicating how to adjust the weight to reduce the loss; Represents the learning rate of the adaptive weight of the residual point, which controls the step size of the adaptive weight update; Represents the loss function L with respect to the adaptive weight of the residual point The gradient of , indicates how to adjust the weights to increase attention to the residual points.

[0042] (4) Loss function definition: Based on the obtained boundary conditions, initial conditions, physical formulas or mathematical expressions, and functional formulas, boundary loss, initial loss, PDE loss, and functional loss are constructed respectively; as shown in the formula below:

[0043] L(w)=λ s L s (w)+λ r L r (w)+λ b L b (w)+λ0L0(w)

[0044] Among them, Ls, Lr, Lb, and L0 represent functional loss, PDE loss, boundary loss, initial loss, and λ respectively. s ,λ r ,λ b , λ0 represent the corresponding weights respectively.

[0045] (5) Network training and result comparison: The physical information neural network is trained by performing forward propagation, calculating the loss value of each part of the loss function, and then performing backpropagation and updating the parameters. After training is completed, the image is plotted and the predicted solution of the neural network is compared with the exact solution to evaluate the performance of the model.

[0046] Example 1: For the mathematical problem of the two-dimensional diffusion equation, the state variable u, the accompanying state variable z, and the control variable q are predicted. The specific details are as follows:

[0047] This embodiment adopts the mathematical expression of the two-dimensional diffusion equation. For details, please refer to the literature "[1] Lv Tong, Ye Xingyang. Numerical analysis of the variable step size BDF2 format for the optimal control problem of a class of linear reaction-diffusion equations [J]. Computational Mathematics, 2025, 47(01): 79-97.". This embodiment adopts the physical information neural network (PINN) method to solve the two-dimensional diffusion problem to avoid the problem that the traditional numerical method significantly increases the amount of calculation due to the increase in grid points. Specifically, this embodiment does not use any real data, and only performs neural network prediction based on the mathematical expression, boundary conditions, initial conditions and functionals of the two-dimensional diffusion problem. The prediction results show that the physical information neural network with an adaptive weight mechanism can solve the two-dimensional diffusion equation problem well, and its generalization is also well reflected.

[0048] Furthermore, the accuracy is superior to that of traditional numerical methods: For a 32×32 grid with k = -1, the L2 error of the traditional numerical method in the paper is: state variable u = 2.31e-03, adjoint state z = 2.95e-03, and control variable q = 2.95e-03. The L2 error of the neural network is: u = 4.7317e-04, z = 2.3633e-04, and q = 8.0243e-04. Overall, the neural network reduces the error by 79.5%, 91.9%, and 72.8%, respectively, compared to the traditional numerical method. This demonstrates the feasibility and superiority of physical information neural networks in solving optimal control problems.

[0049] The method of the present invention is used to predict the result graph of the two-dimensional diffusion equation and its corresponding relative error. Figure 2 As shown in (a) and (b), the accuracy of the neural network can be clearly seen by comparing the neural network prediction solution with the analytical solution.

[0050] Example 2: For the mathematical problem of the three-dimensional diffusion equation, the state variable u is predicted. The specific details are as follows:

[0051] For details, please refer to the literature "[1] Chen Guangnan, Li Deyuan, Wan Zhengsu. Discretization format of three-dimensional diffusion equation based on variational principle [J]. Computational Physics 2003, (04): 291-297. DOI: 10.19596 / j.cnki.1001-246x.2003.04.002." The experimental method of this embodiment is the same as that of embodiment 1, except that the output of the model becomes the state variable u and the input of the model becomes x, y, z, t. The method of the present invention is used to predict the two-dimensional cross-sectional diagram, three-dimensional diagram and the corresponding relative error diagram of the three-dimensional diffusion equation. Figure 3(a), (b), and (c) show this. To visualize the two-dimensional difference plots, we fixed z = 0.5 and selected different time points, t = 0, t = 0.25, and t = 0.5, to perform a detailed comparative analysis of the differences between the exact solution and the neural network's predicted solution. The plot clearly shows that at different time points, the difference between the neural network's predicted solution and the exact solution remains within a small range (1e-2). The three-dimensional plot also demonstrates that the neural network correctly learns this three-dimensional diffusion equation.

[0052] The method of the present invention is used to predict the two-dimensional cross-sectional diagram, three-dimensional diagram and the corresponding relative error diagram of the three-dimensional diffusion equation. Figure 3 As shown in (a), (b) and (c), by comparing the neural network prediction solution with the analytical solution, we can clearly see the accuracy of the physical information neural network and the ability of PINN to solve high-dimensional optimal control problems.

[0053] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may be modified and varied in various ways.

Claims

1. A method for using a physical information neural network for optimal control problems, which can be characterized by Use the following steps to represent: Prepare data and formulas for specific problems: First, gather all the information related to the optimal control problem, such as the state at the beginning of the problem, the conditions on the boundary, the physical formulas and mathematical expressions used to describe the problem, and the functional formulas used to measure the control effect. Building a Neural Network: Open PyCharm and begin building a neural network model. Divide the model into an input layer, hidden layers, and an output layer. Based on the specific problem, determine how much information the model will receive (the number of input elements) and what output it will produce (the number of output elements). Then, based on the complexity of the problem, determine the number of hidden layers and the number of neurons in each layer. Parameter initialization and activation function setting: Initialize the adaptive weight to 1 and select the hyperbolic tangent function (tanh) as the activation function; Loss function definition: Based on the obtained boundary conditions, initial conditions, physical formulas or mathematical expressions, and functional formulas, boundary loss, initial loss, PDE loss, and functional loss are constructed respectively; Network Training and Results Comparison: Train the physical information neural network by performing forward propagation, calculating the loss values ​​for each component of the loss function, and then performing backpropagation to update the parameters. After training is complete, plot the graph and compare the neural network's predicted solution with the exact solution to evaluate the model's performance.

2. The method for using a physical information neural network for an optimal control problem according to claim 1, characterized in that: Physical information neural networks are completely different from other neural networks, which rely on data for training. When solving optimal control problems, physical information neural networks only require boundary conditions, initial value conditions, and the corresponding partial differential equations and functionals.

3. The method for using a physical information neural network for an optimal control problem according to claim 1, characterized in that: Since the input and output of each optimal control problem are different, specific analysis must be carried out for each specific problem.

4. The method for using a physical information neural network for an optimal control problem according to claim 1, characterized in that: First, the initial weight value of each part of the loss function is set to 1; then, during the training process, the weight is dynamically adjusted according to the changing trend of the loss values ​​of different parts - this process is completed by training the network to simultaneously minimize the loss and maximize the weight. In the process of solving the optimal control problem, the weight adaptation mechanism can dynamically adjust the weight of each factor according to the characteristics of the problem, and maintain good generalization capabilities in multiple scenarios. Compared with the manual trial and error method of determining weight parameters, which relies on experience and has randomness and uncertainty, the weight adaptation mechanism, with its systematic parameter optimization logic, has demonstrated significant advantages in control accuracy and efficiency, making it an ideal choice for optimizing control model parameters. In addition, the reasonable selection of activation function is also a key link in solving the optimal control problem.

5. The method for using a physical information neural network for an optimal control problem according to claim 1, characterized in that: The above uses the boundary, initial conditions and physical or functional formulas of the optimal control problem to write out the boundary term loss, initial term loss, partial differential equation loss and functional loss respectively. Finally, the weighted sum of these four losses is used to obtain the total loss function L: L=a b L b +a i L i +a p L p +a f L f Among them, represents boundary loss, initial loss, PDE loss, and functional loss, respectively, and is a weight coefficient that can be adjusted according to the actual situation of the problem. After minimizing the total loss function L, the powerful learning ability of the neural network can be used to solve the optimal control problem.

6. The method for using a physical information neural network for an optimal control problem according to claim 1, characterized in that: When training a neural network, the required specific data is input, and then after the hidden layer is trained, the results are output from the output layer. Finally, the image trained by the neural network is drawn and compared with the accurate image. The difference in the image and the difference in the data calculation can be used to determine the quality of the training.

Images