Method and apparatus for bilayer physical information neural network for PDE constraint optimization

Abstract

A computer-implemented method for solving an optimization problem of partial differential equation (PDE) constraints using a physical information neural network (PINN) is disclosed in which optimization of a state variable corresponding to the solution of the PDE constraints and a control variable corresponding to an optimization objective is decoupled. The method comprises: initializing a weight of the PINN and the control variable, a solution of the PDE constraint being parameterized by the weight of the PINN; respectively calculating a PDE loss related to the PDE constraint and a target function related to the optimization target; under the condition that the weight of the PINN is fixed, updating the control variable through a first learning rate by using gradient descent of the objective function in one iteration; updating the weight of the PINN through a second learning rate by using gradient descent of the PDE loss in the same iteration under the condition that the updated control variable is fixed; and iteratively updating the control variable and the weight of the PINN until convergence, where the updated weight of the PINN in a previous iteration is used to update the control variable in a next iteration.

Description

Technical Field

[0001] Generally speaking, aspects of the present disclosure relate to artificial intelligence, and more specifically, aspects of the present disclosure relate to methods and apparatus for a two-layer physical information neural network for constrained optimization of partial differential equations. Background Art

[0002] Partial differential equation (PDE) constrained optimization (PDECO) aims to optimize the performance of a physical system constrained by a PDE with desired properties. It is an important task in many fields of science and engineering, with a wide range of applications, including shape optimization problems (such as the design of the shape of an aircraft wing in aerodynamics), parameter optimization of channels in heat transfer, parameter optimization of flow control problems, etc.

[0003] To solve the PDECO problem, traditional numerical methods, such as adjoint methods based on the finite element method (FEM), have been studied for decades. However, FEM solvers are usually expensive for high-dimensional problems with large search spaces or large mesh sizes, because the computational cost of FEM grows quadratically to cubically with respect to the mesh size.

[0004] To alleviate this problem, neural networks such as DeepONet have recently been proposed as alternative models to FEM. DeepONet learns the mapping from control (decision) variables to the solution of PDE, and further replaces the PDE constraints with operator networks. However, these methods require pre-training of large operator networks, which is not easy and inefficient. In addition, if the optimal solution is outside the training distribution, its performance may deteriorate.

[0005] Physically-Informed Neural Networks (PINNs) solve learning tasks while obeying the properties of the laws of physics. This is achieved by informing the loss function with the mathematical equations that govern the physical system. The general procedure for solving differential equations with PINNs involves finding the parameters of the network that minimize the loss function, involving the mismatch between the output and the data and the residuals of the boundary and initial conditions, the PDE equations, and any other physical constraints required.

[0006] Therefore, another neural approach is proposed to solve the PDECO problem using a single PINN instead of training a large operator network. It uses the method of Lagrange multipliers to treat the PDE constraints as regularization terms, which can optimize both the objective and the PDE loss. However, such methods introduce a trade-off between optimizing the objective and the regularization term (i.e., the PDE loss), which is critical to the performance. Setting the correct weights for balancing these terms is usually not easy due to the lack of theoretical guidance.

[0007] Therefore, existing methods are inadequate to handle those PDE constraints that have complex or nonlinear dependencies on the optimization objective. Effective strategies for handling PDE constraints to solve PDECO problems must be developed. Summary of the invention

[0008] A simplified overview of one or more aspects is given below to provide a basic understanding of these aspects. This overview is not an extensive overview of all contemplated aspects, and is intended to neither identify key or important elements of all aspects nor to identify the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to a more detailed description that is presented later.

[0009] In general, a novel two-layer optimization framework for solving PDECO problems is disclosed, called a two-layer physical information neural network (BPN) using Broyden's super gradient. The optimization of the objective and PDE constraints is decoupled, thereby naturally solving the challenge of loss balance in regularized methods. In order to solve the two-layer optimization problem, an iterative method is disclosed that optimizes a PINN with PDE constraints in an inner loop while using super gradients to optimize the control variables for the objective function in an outer loop. If the optimization in the inner loop is complex, it is challenging to calculate the super gradients for the control variables in the two-layer optimization. Therefore, a method for calculating super gradients based on implicit differentiation using Broyden's method is also disclosed.

[0010] According to one aspect of the present disclosure, a computer-implemented method for solving a partial differential equation (PDE) constrained optimization problem using a physical information neural network (PINN) is disclosed, wherein the optimization of state variables corresponding to the solution of the PDE constraint and control variables corresponding to the optimization objective are decoupled. The method includes: initializing the weights of the PINN and the control variables, wherein the solution of the PDE constraint is parameterized by the weights of the PINN; calculating the PDE loss associated with the PDE constraint and the objective function associated with the optimization objective respectively; updating the control variables by a first learning rate in one iteration using the gradient descent of the objective function under the condition that the weights of the PINN are fixed; updating the weights of the PINN by a second learning rate in the same iteration using the gradient descent of the PDE loss under the condition that the updated control variables are fixed; and iteratively updating the control variables and the weights of the PINN until convergence, wherein the updated weights of the PINN in the previous iteration are used to update the control variables in the next iteration.

[0011] According to a further aspect, the method includes training the initialized PINN under the initialized control variables for a certain number of rounds as warm-up.

[0012] According to another aspect, the updating of the weights of the PINN further comprises: updating the weights of the PINN for a specific number of rounds using the second learning rate in the same iteration using the gradient descent of the PDE loss under the condition that the updated control variable is fixed.

[0013] According to a further aspect, the gradient descent of the objective function is calculated as a super gradient of the objective function with respect to the control variable based on implicit function theorem differentiation.

[0014] According to further aspects, based at least on calculating the inverse vector-Hessian product as to calculate the super gradient of the objective function with respect to the control variable, where represents the objective function, ε represents the PDE loss, and w represents the weight of the PINN, w * represents the updated weight of the PINN in the previous iteration.

[0015] According to another aspect, by finding the linear equation The root of * to calculate the inverse vector-Hessian product, which further includes iteratively approximating the root z using a low-rank Broyden method * .

[0016] According to another aspect, the low-rank Broyden method is used to iteratively approximate the root z * It also includes: in each iteration, performing the following operations: converting the inverse of the Hessian matrix Approximately where k is B i rank; according to z i+1 =z i -α·B i g i (z) and v i+1 =B i Δz i+1 To update z, u and v, α is the step size; by To update the inverse of the Hessian matrix

[0017] According to another aspect, the low-rank Broyden method is used to iteratively approximate the root z * Also includes: Setting B based on memory limitations i The maximum value of the rank, wherein the maximum value of the rank is less than the maximum number of iterations of the low-rank Broyden method.

[0018] According to another aspect, the low-rank Broyden method is used to iteratively approximate the root z * It also includes: setting a maximum number of iterations of the low-rank Broyden method; and running the iterations of the low-rank Broyden method until the maximum number of iterations is reached or the error is within a threshold.

[0019] A computer-implemented method for solving a partial differential equation (PDE) constrained optimization flow control problem in a physical system using at least one of the methods disclosed herein using a physical information neural network (PINN), wherein the physical system is characterized by the PDE constraints, the state variables of the PDE constraints include at least one of flow rate or pressure, and wherein the optimization goal is to find an optimal input flow distribution, the optimal input flow distribution is parameterized by the control variables to make the output flow uniform, including outputting updated control variables representing the optimal input flow distribution after convergence.

[0020] A computer-implemented method for solving a partial differential equation (PDE) constrained optimization shape optimization problem in a physical system using a physical information neural network (PINN) utilizing at least one of the methods disclosed herein, wherein the physical system is characterized by the PDE constraints, the state variables of the PDE constraints include at least one of flow rate or pressure, and wherein the optimization goal is to find an optimal shape of an object, the optimal shape being parameterized by the control variables, the optimal shape minimizing the resistance or pressure from the flow, including outputting updated control variables representing the optimal shape after convergence. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The disclosed aspects will be described in conjunction with the accompanying drawings, which are provided to illustrate and not to limit the disclosed aspects.

[0022] Figure 1 An exemplary prior art framework for the PDECO problem in accordance with various aspects of the present disclosure is shown.

[0023] Figure 2 An exemplary framework for solving the PDECO problem according to various aspects of the present disclosure is shown.

[0024] Figure 3 Details of an exemplary framework for solving the PDECO problem according to various aspects of the present disclosure are shown.

[0025] Figure 4 An exemplary flow chart for solving a PDECO problem according to various aspects of the present disclosure is shown.

[0026] Figure 5 An exemplary computing system in accordance with various aspects of the present disclosure is shown. DETAILED DESCRIPTION

[0027] The present disclosure will now be discussed with reference to several example implementations. It will be understood that these implementations are discussed only to enable those skilled in the art to better understand and thus implement embodiments of the present disclosure, rather than implying any limitation on the scope of the present disclosure.

[0028] Various embodiments will be described in detail with reference to the accompanying drawings. Wherever possible, the same reference numerals will be used throughout the drawings to refer to the same or similar parts. Reference to specific examples and embodiments is for illustrative purposes and is not intended to limit the scope of the present disclosure.

[0029] Partial differential equation (PDE) constrained optimization (PDECO) aims to optimize the performance of a physical system constrained by a PDE with desired properties. The goal or objective of PDECO will usually be parameterized by a set of control variables, and the state variables of the physical system will correspond to the solutions to the PDE constraints.

[0030] Substitute modeling is an important method for PDECO. Physical information neural network (PINN) is a powerful and flexible substitute for representing the solution of PDE. Some existing methods (such as hPINN) regard PDE constraints as regularization terms and optimize control variables and state variables simultaneously. Such methods use penalty methods and Lagrangian methods to adjust the weight of multipliers. Some of these methods use the same formula, but use linear search to find the maximum weight when the PDE error is within a certain range. The key limitation of these methods is that the weight of the multiplier is determined heuristically, which may be suboptimal and unstable.

[0031] Another class of methods trains an operator network from the control variables to the solution or objective function of the PDE. Several methods use a grid-based approach and predict the state at all grid points from the control variables simultaneously. As another example, PI-DeepONet adopts the architecture of DeepONet and uses physical information loss (also known as PDE loss) to train the network.

[0032] However, all existing methods produce unsatisfactory results if the optimal solution is out of distribution. To better illustrate our proposed method, we first formulate the PDECO problem as follows.

[0033] Let Y, U, and V be three Banach spaces. The solution domain of a PDE is called the state variable, i.e. And the function or variable that can be controlled is the control variable, that is, where Y ad and U ad is the admissible space. PDECO can be formulated as: in is the objective function, and e:Y×U→V is the PDE constraint. In general, a PDE system with e(y,u)=0 contains multiple equations and boundary conditions / initial conditions, as shown below: in is a differential operator representing the PDE, and represents boundary conditions / initial conditions.

[0034] As an example, for a flow control problem, the goal is to find the best input flow distribution so that the output flow is as uniform as possible. The state variables represent the state of the physical system, which can usually be flow rate and pressure, etc., and the control variables can be parameters that can parameterize the flow distribution. As another example, for a shape optimization problem, the goal is to find the best shape that minimizes the resistance from the flow, the state variables can also be flow rate and pressure, but the control variables are the parameters of the structure to be optimized.

[0035] As mentioned above, existing regularization-based methods (e.g., penalty methods) solve the PDECO problem by minimizing the following objective: The solution y and the control variable u are respectively given by and parameterization, where w is the weight of the PINN. is a hyperparameter that balances these terms in the optimization objective. A major difficulty is that i Difficult to set up and sensitive to results due to the complex nature of the regularization term (PDE constraints). In general, large λ2 makes it difficult to optimize the objective The small λ i May cause w In addition, the optimal λ may change at different stages of training.

[0036] Figure 1 An exemplary prior art framework for the PDECO problem in accordance with various aspects of the present disclosure is shown.

[0037] At the beginning, an initial set of weights w of the PINN and control variables θ are input to box 101. At box 101, the objective function representing the goal to be optimized can be calculated as Its governing PDE constraint is and They represent the boundary / initial conditions of the physical system. As mentioned above, the solution of the PDE can be represented by the state variables y, and the objective function The control variable of can be represented by u, and the state variable and control variable are parameterized by w and θ respectively.

[0038] At block 102, utilizing the efficient λ i The objective function and the regularization term corresponding to the PDE constraint to form As shown in Equation (3). Then, by making the combined objective function Minimize to optimize the control variables and state variables simultaneously. Then, output the optimized w and θ.

[0039] Figure 1 These are shown only as examples, and other examples would be possible.

[0040] To address the above challenges of regularization-based methods, this paper discloses a method to interpret PDECO as a two-level optimization problem, which can inevitably promote new solvers. Specifically, the following two-level optimization problem is solved:

[0041] In the outer loop, given w * The optimal value of is minimized, and then the PDE loss is optimized using PINN in the inner loop with fixed θ. The objective ε of the inner loop subproblem is:

[0042] By converting the problem in Eq. (1) into a two-level optimization problem as described above, the optimization of the state variables and the control variables of the PDE can be decoupled, which alleviates the difficulty of setting λ in Eq. (3). i The trouble with finding appropriate hyperparameters for .

[0043] To solve the two-level optimization problem proposed in Equation (4), the inner loop and the outer loop are designed to be executed iteratively. Figure 2 An exemplary framework for solving the PDECO problem according to various aspects of the present disclosure is shown.

[0044] The objective function representing the goal to be optimized is Its governing PDE constraint is and They represent the boundary / initial conditions of the physical system. As mentioned above, the solution of the PDE can be represented by the state variables y, and the objective function The control variable of can be represented by u, and y and u are parameterized by w and w respectively.

[0045] like Figure 2 As shown, the initial set of weights w of the PINN and the control variable θ are input to box 201. At box 201, the higher layer calculates And optimize θ, where in one iteration given a fixed value of w makes is minimized with respect to θ. The optimized θ is then passed to block 202 for optimizing w.

[0046] At block 202, the PINN is trained in the lower layers using only the PDE loss ε as in equation (5). Afterwards, the optimized w will be transmitted back to block 201 to optimize θ in the next iteration. Then after a certain number of iterations and / or convergence, the optimized w and θ are output. By decoupling the optimization of state variables and control variables, the shortcomings of the above-mentioned regularization-based methods can be overcome.

[0047] Figure 2 These are shown only as examples, and other examples would be possible.

[0048] Two-level optimization is widely used in various machine learning tasks, such as neural architecture search, meta-learning, and hyperparameter optimization. One of the key challenges is to calculate the super gradient with respect to the inner loop optimization. Some previous methods use unfolded optimization or truncated unfolded optimization, which will differentiate the optimization process. However, this is not scalable if the inner loop optimization is a complex process. Some other methods calculate super gradients based on implicit function theorems. This requires calculating the inverse Hessian vector product (inverse-HVP). In some previous methods, it is proposed to use Newman series to approximate super gradients. Some works also use conjugate gradient methods. The approximation of implicit differentiation is crucial to the accuracy of super gradient calculation.

[0049] To optimize the control variables in the outer loop or higher level, such as equations (4) and Figure 2 What is being discussed is to calculate The gradient of the parameter with respect to the control variable θ is called the super gradient. The super gradient is calculated based on the differentiation of the implicit function theorem, which requires the calculation of highly complex inverse Hessian-Jacobian products. To solve this problem, the use of Broyden's method is proposed, which provides an efficient approximation at a superlinear convergence rate, which will be discussed in detail below.

[0050] Higher level goals Depends on the optimal w optimized by the lower layers * , as shown in equation (4) and Figure 2 What is discussed is, Therefore, it is necessary to consider w when calculating the super gradient * The Jacobian matrix with respect to θ. Since w * minimizes the lower level problem, so it can be derived by applying the Cauchy implicit function theorem As shown below: Theorem 1: If for some (w′,θ′), the lower level optimization is solved, that is and is reversible, then there exists a function w around (w′,θ′) * =w * (θ), so that and:

[0051] By Theorem 1, the super gradient can be calculated analytically as:

[0052] However, computing the inverse of the Hessian matrix The parameters of the neural network are difficult to solve. To address this challenge, we can first calculate It is also known as the inverse vector-Hessian product. As mentioned earlier, some previous work uses the Newman series to approximate the inverse of the Hessian matrix. However, this approach is a rough and inaccurate estimate of the super gradient.

[0053] Therefore, this paper discloses a more efficient and effective method for computing the inverse vector-Hessian product, which enjoys superlinear convergence rate. * is equivalent to finding the roots of the following linear equation: Note that for g w Each evaluation of (z) requires only two Jacobian vector products to be calculated, which does not require the creation of an instance of the Hessian matrix as in the prior art, resulting in low computational cost. Specifically, a low-rank Broyden method is used to iteratively approximate the solution z * In each iteration, we first The inverse approximation of is:

[0054] Then, z, u, and v are updated according to the following rules: z i+1 =z i -α·B i g i (z) (11) v i+1 =B2Δz i+1 (13) where Δz i+1 =z i+1 -z i , Δg i+1 =g i+1 -g i, and α is the step size. In one example, α may be set to 1, and other examples are allowed.

[0055] In summary, Hessian's inverse can be updated as follows:

[0056] After m iterations, z m Used as z * Approximation of .

[0057] Based on equation (10), it can be seen that B i Through the column vector u k and v k This means that instead of storing The entire high-dimensional matrix only needs to record u k and v k , which are all vectors with dimension k, where k=1…K and K is a tunable parameter depending on memory limitations. Therefore, will produce a matrix k with rank k i , k will be much smaller than dimensionality in order to achieve low computational cost.

[0058] Run the Broyden iteration until the maximum iteration is reached or the error is within a threshold. Note that the maximum memory limit should be less than the maximum number of iterations in order to approximate the inverse of the Hessian as a K-rank matrix. When the Broyden iteration is run more than K times, only the K most recent u are kept due to memory limitations. k and v k .

[0059] The effects of different iterations and memory limits of the Broyden method are investigated by experiments on the Heat 2d problem and the NS backstep problem. The results for the Heat 2d equation are shown in Table 1. It can be seen that the performance generally improves (the objective function decreases) with increasing maximum memory limit and maximum iteration. In this problem, a rank of more than 16 for approximating the inverse Hessian can handle the problem. If more than 32 iterations are used, there is no significant gain. Table 1 - Ablation study on the Heat2d problem. Lower scores mean better performance.

[0060] The results for the NS backstep problem are shown in Table 2. The trend is similar to the Heat2d problem, and performance generally improves with increasing Broyden iterations. However, it can also be seen that if only 8 memory steps are used, the problem becomes more difficult and performance deteriorates even with many iterations. Table 2 - Ablation study on the NS back-step problem. Lower scores mean better performance.

[0061] Figure 3 Details of an exemplary framework for solving the PDECO problem according to various aspects of the present disclosure are shown.

[0062] Similar to Figure 2 , an initial set of weights w0 of the PINN and control variables θ0 may be input to block 301. Alternatively, the initial PINN may be trained under the control variables θ0 for a specific number of epochs N w As a warm-up, and then the pre-trained PINN and its weights may be input to block 301 .

[0063] At block 301, an objective function having an expression similar to equation (1) is calculated: Its governing PDE constraint is and They represent the boundary / initial conditions of the physical system. As mentioned above, the solution to the PDE constraints can be represented by the state variables y, and the objective function The control variable of can be represented by u, and y and u are parameterized by w and θ respectively.

[0064] To optimize θ by gradient descent with fixed w in the outer loop, we need to calculate Gradient with respect to θ. As discussed above and shown in block 301, the super gradient of θ is calculated using IFT differentiation, and the inverse vector-Hessian product is further calculated based on Broyden's method, as in equations (9) and (14), which uses a low-rank approximation for acceleration. As shown in block 301 using solid blocks, the high-dimensional matrix is ​​replaced by a low-rank vector-matrix product.

[0065] After optimizing θ in the iteration, the optimized θ is transmitted to block 302 to fine-tune only the PINN using the PDE loss as in equation (3). As shown in block 302, the PINN is trained using the gradient descent of the PDE loss under the condition that the optimized θ is fixed. In the example, the PINN can be trained for a specific number of rounds N f To obtain the best * .

[0066] After optimizing w in iterations, the optimized w * is passed back to block 301 to optimize θ in the next iteration. These two steps are performed iteratively until convergence.

[0067] The number of warm-up rounds N for the inner loop of PINN is also studied experimentally. w and different fine-tuning rounds N fThe Heat 2d problem is selected and the results are shown in Tables 3 and 4. As can be seen from the table, these hyperparameters have a considerable impact on the convergence speed. However, their impact on the final performance is small in a large range. Therefore, BPN is robust to the choice of these two parameters on the Heat 2d problem, but by choosing a moderate N w and / or N f More efficient and effective results can be achieved. Table 3 - Ablation study on the impact of fine-tuning rounds on PINN Table 4 - Ablation study of the impact of warm-up rounds on PINN

[0068] Based on the previous description and examples, the disclosed method as a whole is named as a two-layer physical information neural network (BPN) using Broyden super gradient, however, one or more aspects can be implemented alone or combined with their corresponding effects. The pseudo code of BPN is outlined in Algorithm 1.

[0069] The disclosed BPN first has the idea of ​​solving the transposed linear system in equation (9) to reduce computational cost. In addition, compared with the constrained optimization in the existing methods, the disclosed two-level optimization is a more general framework. It allows more flexible design choices, such as replacing the FEM solver with PINN. In addition, equation (9) solves the system in the parameter space of the neural network, but some traditional methods use equations corresponding to the real domain defined on the admissible space or grid.

[0070] Figure 4 An exemplary flow chart for solving the PDECO problem according to various aspects of the present disclosure is shown. As described below, some or all of the features shown may be omitted in a specific implementation within the scope of the present disclosure, and some of the features shown may not be needed to implement all embodiments. In addition, some frames may be performed in parallel or in different orders. In some examples, the method may be performed by any suitable device or unit for performing the function or algorithm described below.

[0071] The PDECO problem aims to optimize the performance of a physical system constrained by a PDE with desired properties, which may be a variety of problems in science and engineering, to name a few, flow control problems, shape optimization problems, drag minimization, etc. Shape optimization problems or drag minimization problems aim to design a structure such as an airfoil (given the shape, size and distribution of materials) with high performance for a system characterized by a PDE. Figure 4The described approach can be applied to all PDECO problems with appropriate PDE constraints and is not limited to the following examples.

[0072] For example, the Naiver-Strokes equation is one of the most important equations in fluid mechanics, aerodynamics and applied mathematics, which is difficult to solve due to its high nonlinearity. In this problem, the NS equations are solved in a pipeline, and the goal is to find the optimal input flow distribution f(y) to make the output flow as uniform as possible. The velocity field is u = (u, c), and the pressure field is p, and they are confined on a rectangular domain Ω, for example Ω = [0, 1.5] × [0, 1.0]. Two inputs and two outputs and several walls can be defined for this domain, for example:

[0073] The whole problem will be as follows, which can be a concrete expression of equations (1) and (2): Satisfaction u=0,(x,y)∈Γ w

[0074] The objective function is a parabolic function and the velocity field at the second input and output is v2(x)=18(x-0.5)(1-x). In this problem, the Reynolds number can be set to 100, and f(y) can be initialized identically to the objective function, f(y)=4y(1-y). The problem can be solved using the disclosed method as a bi-level optimization problem described below, by solving the PDE constraints using PINN and minimizing J with respect to f given the optimal solution to the PDE constraints.

[0075] As another example, back-step flow is a classic example of a flow that may exhibit turbulence. In this problem, the goal is also to find the optimal input flow distribution f(y) so that the output flow is symmetric and uniform. The geometry of the back-step can be viewed as the union of two rectangles (such as Ω = [0,1] × [0,0.5] ∪ [1,2] × [0,1]). The input is the left side of the region, and the output is the right side of the region, Γ in ={(0,y):0≤y≤0.5} Γ out ={(2,y):0≤y≤1}

[0076] The output velocity field is to be optimized. Satisfaction u=(f(y),0),(x,y)∈Γ in u=0,(x,y)∈Γ w p=0,(x,y)∈Γ out

[0077] The target velocity field is And the input speed f(y) is initialized to 8y(0.5-y). This problem can also be solved using the disclosed method as a two-level optimization problem described below, by solving the PDE constraints using PINN and minimizing J with respect to f given the optimal solution of the PDE constraints.

[0078] For other types of shape optimization problems, for example, minimizing the drag on an obstacle for the NS equations, this is a shape optimization task to find the best shape of the obstacle that minimizes the drag from the flow. The input is the left side of the region, and the output is the right side of the region, shown as: Γ in ={(0,y):0≤y≤8} Γ out ={(8,y):0≤y≤8}

[0079] In the domain Ω=[0,8] 2 \Ω o The flow field is defined on the o is an obstacle. The shape of the obstacle is an ellipse parameterized by parameters a∈[0.5,2.5]. The goal is to minimize the following objective: Satisfaction u=0,(x,y)∈Γ w p=0,(x,y)∈Γ out

[0080] This problem can also be solved using the disclosed method as a bi-level optimization problem to be described below, by solving the PDE constraints using PINN and minimizing J with respect to f given the optimal solution to the PDE constraints.

[0081] Now, Figure 4The disclosed method is generally applicable to solving the PDECO problem. The method starts at block 401, where PINNu is initialized. w The weight w0 of PINN and the control variable θ0, where the solution of the PDE constraint is parameterized by the weight w of PINN. As an example, the PINN is initialized with random parameters w0, and the control variable θ0 is initialized with a guess.

[0082] The method proceeds to block 402, where a PDE loss associated with the PDE constraint and an objective function associated with the optimization goal are calculated, respectively. As an example, the PDE loss ε associated with the PDE constraint can be calculated according to equation (5). Referring to the several PDECO problems mentioned above, the PDE constraint can be a boundary and / or initial condition of the physical system. In addition, as an example, the objective function associated with the optimization goal can be calculated using a form similar to equation (1).

[0083] The method proceeds to box 403, where the initialized PINN is trained under the initialized control variables for a specific number of rounds as a warm-up, as shown by the dotted line, and box 403 is an optional box. The warm-up will have a considerable impact on the convergence speed, but has a small impact on the final performance, so the number of rounds can be selected based on the implementation, without limitation.

[0084] The method proceeds to block 404 where the control variables are updated by a first learning rate using a gradient descent of the objective function in one iteration under the condition that the weights of the PINN are fixed.

[0085] In the example, the gradient descent of the objective function is calculated as the super gradient of the objective function with respect to the control variable based on the implicit function theorem differentiation. For example, the super gradient of the objective function with respect to the control variable can be calculated according to Theorem 1 according to equation (8).

[0086] As another example, based at least on computing the inverse vector-Hessian product as To calculate the super gradient of the objective function with respect to the control variable, represents the objective function, ε represents the PDE loss, and w represents the weight of PINN, w * represents the updated weights of the PINN in the previous iteration.

[0087] As another example, by finding the linear equation The root of * to compute the inverse vector-Hessian product, which further includes iteratively approximating the root z using a low-rank Broyden method * . In order to iteratively approximate the root z * , in each iteration of the low-rank Broyden method, it also includes: the inverse of the Hessian matrix Approximately where k is B i rank; according to z i+1 =z i -α·B i g i (z) and v i+1 =B i Δz i+1 To update z, u and v, α is the step size; by To update the inverse of the Hessian matrix

[0088] As another example, to iteratively approximate the root z * , further comprising: setting B based on memory limitations i The maximum value of the rank, where the maximum value of the rank is less than the maximum number of iterations of the low-rank Broyden method.

[0089] As another example, to iteratively approximate the root z * , and also includes: setting a maximum number of iterations of the low-rank Broyden method; running iterations of the low-rank Broyden method until the maximum number of iterations is reached or the error is within a threshold.

[0090] After the control variables are updated, the method proceeds to block 405, where the weights of the PINN are updated by a second learning rate using a gradient descent of the PDE loss in the same iteration under the condition that the updated control variables are fixed. As an example, the PINN is trained for a specific number of rounds using the updated control variables.

[0091] After determining at block 406 that the PINN is optimized and has not converged, the optimal weights of the PINN are transmitted back to block 404 for the next iteration. Figure 4 It can also be seen that the operations of blocks 404 and 405 form an outer loop of optimization of the control variables, and the operations in block 405 itself are inner loop optimizations for fine-tuning the PINN, which are also implemented by Figure 3 Also, in one aspect, in block 404 there is a Broyden method iteration.

[0092] After optimizing and determining that the PINN has converged at box 406 , the optimized control variables and the PINN are output at box 407 .

[0093] Figure 5An exemplary computing system according to various aspects of the present disclosure is shown. The computing system may include at least one processor 510. The computing system may also include at least one storage device 520. It should be understood that the storage device 520 may store computer executable instructions that, when executed, cause the processor 510 to perform operations according to the embodiment of the present disclosure. Figures 1 to 4 Any operation of the embodiments of the present disclosure described.

[0094] Embodiments of the present disclosure may be embodied in a computer-readable medium such as a non-transitory computer-readable medium. The non-transitory computer-readable medium may include a computer-implemented method that, when executed, causes one or more processors to perform a method for solving a partial differential equation (PDE) constrained optimization problem using a physical information neural network (PINN), wherein optimization of state variables corresponding to solutions to the PDE constraints and control variables corresponding to the optimization objective are decoupled. The method includes: initializing the weights of the PINN and the control variables, wherein the solution of the PDE constraint is parameterized by the weights of the PINN; respectively calculating the PDE loss associated with the PDE constraint and the objective function associated with the optimization objective; updating the control variables by using a first learning rate in one iteration using the gradient descent of the objective function under the condition that the weights of the PINN are fixed; updating the weights of the PINN by using a second learning rate in the same iteration using the gradient descent of the PDE loss under the condition that the updated control variables are fixed; and iteratively updating the control variables and the weights of the PINN until convergence, wherein the updated weights of the PINN in the previous iteration are used to update the control variables in the next iteration.

[0095] Embodiments of the present disclosure may be embodied in a computer-readable medium such as a non-transitory computer-readable medium. The non-transitory computer-readable medium may include a computer-implemented method that, when executed, causes one or more processors to perform a method for solving a partial differential equation (PDE)-constrained optimization flow control problem in a physical system using a physical information neural network (PINN) using at least one of the methods disclosed herein, wherein the physical system is characterized by the PDE constraints, the state variables of the PDE constraints include at least one of flow rate or pressure, and wherein the optimization goal is to find an optimal input flow distribution, the optimal input flow distribution being parameterized by the control variables to make the output flow uniform, including: outputting updated control variables representing the optimal input flow distribution after convergence.

[0096] Embodiments of the present disclosure may be embodied in a computer-readable medium such as a non-transitory computer-readable medium. The non-transitory computer-readable medium may include instructions that, when executed, cause one or more processors to perform operations.

[0097] The non-transitory computer readable medium may also include, when executed, causing one or more processors to perform the Figures 1 to 4 Instructions for any operation of the described embodiments of the present disclosure.

[0098] It should be understood that all operations in the above methods are merely exemplary operations, and the present disclosure is not limited to any operations in these methods or the order of these operations, and should encompass all other equivalents under the same or similar concept.

[0099] It should also be understood that all modules in the above-mentioned device can be implemented in various ways. These modules can be implemented as hardware, software or a combination thereof. In addition, any module in these modules can be further divided into submodules or combined together in function.

[0100] The previous description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Therefore, the claims are not intended to be limited to the aspects shown herein. All structural and functional equivalents of the elements of the various aspects described throughout this disclosure are expressly incorporated herein by reference and are intended to be covered by the claims, and these structural and functional equivalents are known or will be known later to those of ordinary skill in the art.

Claims

1. A computer-implemented method for solving partial differential equation (PDE) constrained optimization problems using a physical information neural network (PINN), wherein: Optimization of state variables corresponding to solutions of PDE constraints and control variables corresponding to optimization objectives are decoupled, and the method includes: Initializing weights of the PINN and the control variables, wherein the solution of the PDE constraint is parameterized by the weights of the PINN; Calculating the PDE loss associated with the PDE constraint and the objective function associated with the optimization goal respectively; Under the condition that the weight of the PINN is fixed, updating the control variable by using a first learning rate in one iteration using a gradient descent of the objective function; Under the condition that the updated control variable is fixed, updating the weight of the PINN by a second learning rate using the gradient descent of the PDE loss in the same iteration; and The control variables and the weights of the PINN are iteratively updated until convergence, wherein the updated weights of the PINN in a previous iteration are used to update the control variables in a next iteration.

2. The computer-implemented method of claim 1 , further comprising: The initialized PINN is trained under the initialized control variables for a specific number of epochs as warm-up.

3. The computer-implemented method of claim 1 , wherein updating the weight of the PINN further comprises: Under the condition that the updated control variable is fixed, the weight of the PINN is updated for a specific number of rounds using the second learning rate using the gradient descent of the PDE loss in the same iteration.

4. The computer-implemented method of claim 1 , wherein: The gradient descent of the objective function is calculated as a super gradient of the objective function with respect to the control variable based on implicit function theorem differentiation.

5. The computer-implemented method of claim 4, wherein: At least based on computing the inverse vector-Hessian product as to calculate the super gradient of the objective function with respect to the control variable, where represents the objective function, ε represents the PDE loss, and w represents the weight of the PINN, w * represents the updated weight of the PINN in the previous iteration.

6. The computer-implemented method of claim 5, wherein: By finding the linear equation The root of * to compute the inverse vector-Hessian product, and also: The root z is iteratively approximated using a low-rank Broyden method * .

7. The computer-implemented method of claim 6, wherein: The root z is iteratively approximated using the low-rank Broyden method * Also includes: In each iteration, do the following: The inverse of the Hessian matrix Approximately where k is B i rank; According to z i+1 =z i -α·B i g i (z) and v i+1 =B i Δz i+1 To update z, u and v, α is the step size; pass To update the inverse of the Hessian matrix 8. The computer-implemented method of claim 7, wherein: Iteratively approximating the root z using the low-rank Broyden method further comprises: Set B based on memory limitations i The maximum value of the rank, wherein the maximum value of the rank is less than the maximum number of iterations of the low-rank Broyden method.

9. The computer-implemented method of claim 7, using the low-rank Broyden method to iteratively approximate the root z further comprising: Setting the maximum number of iterations of the low-rank Broyden method; Iterations of the low-rank Broyden method are run until the maximum number of iterations is reached or the error is within a threshold.

10. A computer-implemented method for solving a partial differential equation (PDE) constrained optimization flow control problem in a physical system using a physical information neural network (PINN) according to any one of the methods of claims 1-9, wherein: The physical system is characterized by the PDE constraint, the state variables of the PDE constraint include at least one of flow rate or pressure, and wherein the optimization goal is to find an optimal input flow distribution parameterized by the control variables to make the output flow uniform, comprising: Updated control variables representing the optimal input flow distribution are output after convergence.

11. A computer-implemented method for solving a partial differential equation (PDE) constrained optimization shape optimization problem in a physical system using a physical information neural network (PINN) according to any one of the methods of claims 1 to 9, wherein: The physical system is characterized by the PDE constraint, the state variables of the PDE constraint include at least one of flow rate or pressure, and wherein an optimization goal is to find an optimal shape of an object, the optimal shape parameterized by the control variables, the optimal shape minimizing the resistance or pressure from the flow, comprising: Updated control variables representing the optimal shape are output after convergence.

12. A computer system comprising: one or more processors; as well as One or more storage devices storing computer-executable instructions which, when executed, cause the one or more processors to perform the operations of the method according to any one of claims 1-9.

13. One or more computer-readable storage media storing computer-executable instructions that, when executed, cause one or more processors to perform the operations of the method of any one of claims 1-9.

14. A computer program product comprising computer executable instructions which, when executed, cause one or more processors to perform the operations of the method according to any one of claims 1-9.

Images