Physical information neural network optimization method based on global pattern regularization
By introducing a global pattern regularization loss function, a global pattern correlation is established, which solves the problem of propagation failure in traditional PINNs and achieves efficient and accurate partial differential equation solving, applicable to various types of partial differential equations.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-29
- Publication Date
- 2026-04-14
AI Technical Summary
Traditional physical information neural networks (PINNs) suffer from propagation failure during training, leading to difficulties in training convergence and low accuracy when solving complex partial differential equations. Furthermore, existing improvement methods have failed to effectively overcome local constraints, increasing training costs.
A global pattern regularization loss function is constructed, and long-range connections between the boundary/initial region and the internal region are established through global pattern correlation derivation. This replaces the traditional local differential operator, optimizes the loss function landscape, and enables rapid propagation of the global solution.
It significantly improves solution accuracy and training efficiency, especially in high-difficulty partial differential equations with low relative error and shortened training time. It is applicable to various types of partial differential equations, including linear and nonlinear equations.
Smart Images
Figure CN121859947A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the interdisciplinary field of artificial intelligence and scientific computing, specifically involving a physical information neural network optimization method based on global pattern regularization, which is used to efficiently solve partial differential equations. It is particularly suitable for solving the propagation failure problem in the training of traditional physical information neural networks (PINNs), and can be widely applied to scientific and engineering scenarios that rely on solving partial differential equations, such as fluid dynamics, medical imaging, and climate modeling. Background Technology
[0002] Physical Information Neural Networks (PINNs), as a novel numerical solution method integrating deep learning and physical laws, can directly learn continuous solutions to partial differential equations by incorporating partial differential equation constraints, boundary conditions, and initial conditions into the loss function, demonstrating great potential in the field of scientific computing. However, traditional PINNs face insurmountable technical bottlenecks in practical applications: their residual loss relies on local differential operators, applying local physical constraints only at each collocation point, resulting in extremely slow propagation of the solution from the boundary / initial region into the spatiotemporal domain, i.e., the "propagation failure" problem.
[0003] In scenarios involving complex coefficients or challenging partial differential equations, propagation failure can lead to difficulties in training convergence and low solution accuracy. For example, when the coefficients of the partial differential equation are large, traditional PINNs, even after extensive iterative training, cannot effectively propagate the solution throughout the entire spatiotemporal domain. Predictions in some regions remain in their initial state, making it difficult to meet the accuracy and efficiency requirements of engineering applications.
[0004] To address these issues, existing technologies primarily focus on improvements such as weighted adjustment of the loss term, optimization domain decomposition using collocational sampling strategies, and activation function improvements. However, these methods remain fundamentally constrained by local constraints and are still limited by local propagation mechanisms, struggling to overcome propagation failures when dealing with complex partial differential equations. Furthermore, while some methods improve performance to some extent by introducing complex optimization strategies or additional computational steps, they significantly increase training costs and reduce solution efficiency.
[0005] Therefore, there is an urgent need for a physical information neural network optimization method that can break through local constraints, establish global connections, and accelerate the propagation of solutions, so as to improve training efficiency while ensuring solution accuracy, especially for solving highly difficult partial differential equations. Summary of the Invention
[0006] 1. Technical Solution
[0007] The core idea of this invention is to utilize the global pattern association inherent in partial differential equations, which is independent of specific boundary / initial conditions, to construct a global pattern regularization (GPR) loss function. This function completely replaces the local differential operator residual loss in traditional PINNs. By establishing direct long-range connections between the boundary / initial region and the internal region, the propagation mechanism of the solution is fundamentally changed, accelerating the propagation of the global solution. At the same time, the loss function landscape is optimized, overcoming the optimization failure problem of high-difficulty equations.
[0008] This method specifically includes three core steps: global pattern association derivation, global pattern regularization (GPR) loss function construction, and GPR-based PINNs training and solution. It covers the entire process of offline preparation, model training and online solution, and is applicable to various partial differential equation scenarios, including linear and nonlinear, bounded and unbounded domains.
[0009] (1) Global schema association derivation
[0010] For different types of partial differential equations (linear / nonlinear, bounded / unbounded), based on the finite difference discretization method, the long-range correlations between discrete points in the spatiotemporal domain are derived:
[0011] Linear partial differential equations
[0012] • Unbounded spacetime domain (taking the convection equation as an example): The convection equation is a commonly used linear hyperbolic partial differential equation in fluid dynamics, and its standard form is: , where β is the convection coefficient, which determines the speed and direction of disturbance propagation.
[0013] First, the spatiotemporal domain is discretized: the time domain is... Discretized Time points The time step is ; the spatial domain [0, 2 Discretized into spatial points The space step size is .
[0014] By discretizing the time derivative using a forward difference scheme and the spatial derivative using a backward difference scheme, we obtain the finite difference discretized form of the convection equation: ,in Representing discrete points The approximate solution at that point. After simplification, the recursive relationship of the solution vectors for adjacent time layers can be obtained: .in It is the Courant-Friedrichs-Lewy (CFL) number, used to ensure the stability of the discrete scheme.
[0015] The above recurrence relation can be expressed in matrix-vector form. ,in Let A be the solution vector of the nth time level. The incidence matrix. For periodic boundary conditions, the structure of matrix A is:
[0016]
[0017] Based on the matrix mapping relationship between adjacent time layers, the long-range correlation between any two time layers p>q can be further derived: ,in The pq power of the incidence matrix A can be pre-calculated using matrix exponentiation.
[0018] • Bounded spatial domain (taking the two-dimensional Poisson equation as an example): The two-dimensional Poisson equation is a typical elliptic partial differential equation, widely used in the calculation of physical fields such as electric and gravitational fields. Its standard form is: , in For the Laplace operator, For source terms, It is a bounded square domain.
[0019] Discretize the spatial domain: Discretize both the x-axis and y-axis into N points, with a spatial step size of... discrete points The approximate solution at is expressed as Discretizing the Laplace operator using the central difference scheme yields:
[0020] After sorting, we get: .
[0021] The discrete points are divided into boundary points and interior points: boundary point vector. Includes all discrete point solutions on the domain boundary Interior point vectors Includes all discrete point solutions within the domain (e.g., 1) (points), source term vector The source term values correspond to the interior points. Based on the above discretized equations, a linear relationship between interior points and boundary points can be established: Here, A and B are coefficient matrices. The diagonal elements of A are -4, and the off-diagonal elements correspond to the relationships between adjacent points of the interior points (the elements corresponding to adjacent interior points are 1). The elements of B correspond to the relationships between interior points and boundary points (the elements corresponding to adjacent boundary points are -1). For bounded domains with complex boundary geometry, the above discretized equations can be extended using interpolation methods to establish long-range relationships between interior points and boundary points.
[0022] Nonlinear partial differential equations
[0023] For nonlinear equations (such as reaction equations), the nonlinear recursive relationship of the solution is obtained through finite difference discretization, and then the long-range nonlinear correlation between arbitrary time layers (such as...) is derived. (where G is a predefined nonlinear function).
[0024] The aforementioned long-range correlations are independent of specific boundary / initial conditions and can be calculated in advance during the offline phase, providing a basis for global pattern regularization.
[0025] (2) Construction of global pattern regularization loss function
[0026] Based on the derived long-range correlation, a global pattern regularization loss function is designed to replace the residual loss in traditional PINNs. This loss function constrains the neural network prediction results to satisfy global long-range correlation, forcing the solution to propagate rapidly throughout the entire spatiotemporal domain. Its specific form is determined according to the type of partial differential equation.
[0027] Unbounded linear equations: ,in , These are the solution vectors for the initial time layer and the p-th time layer predicted by the neural network, respectively. This is the power of the pre-calculated long-range correlation matrix.
[0028] Bounded linear equations (such as the two-dimensional Poisson equation): ,in , These are the predicted solution vectors for the interior points and the boundary points, respectively.
[0029] Nonlinear equations (such as reaction equations): ,in This represents the p-th composite of a nonlinear function G.
[0030] (3) Training and solving of PINNs based on global pattern regularization
[0031] Offline preparation phase: Discretize the spatiotemporal domain of the target partial differential equation, calculate the long-range correlation matrix (e.g., A, B) or nonlinear correlation function (e.g., G), and pre-calculate matrix powers (e.g., ...). , (etc.) to reduce computational overhead during online training.
[0032] Network training phase: A fully connected neural network is constructed as the base model, using "global pattern regularization loss + boundary condition loss + initial condition loss" as the total loss function. The Adam optimizer is used for training, minimizing the total loss to update the network parameters. Training does not rely on local differential operators; global constraints accelerate solution propagation.
[0033] Online solution phase: The trained model is applied to solve the objective partial differential equation. A continuous solution can be obtained by directly inputting the spatiotemporal coordinates. For different boundary / initial conditions, it is only necessary to retrain the boundary / initial condition loss parameters or adjust the long-range correlation constraints according to the new conditions, without retraining the entire network.
[0034] 2. Technical Effects
[0035] This invention, by introducing global pattern regularization, completely eliminates the dependence of traditional PINNs on local differential operators, establishes long-range connections between the boundary / initial region and the internal region, significantly accelerates solution propagation, and effectively overcomes the propagation failure problem. Experimental verification shows that this method has the following advantages:
[0036] (1) Higher solution accuracy: In linear and nonlinear partial differential equations, especially in high-difficulty scenarios where traditional methods are difficult to solve, this method has relatively higher accuracy. The error is significantly lower than that of existing methods, enabling high-precision solutions.
[0037] (2) Higher training efficiency: For linear partial differential equations, the training time is shortened by 2-3 times compared with existing methods; for nonlinear partial differential equations, the training efficiency is comparable to existing methods, and the convergence speed is faster.
[0038] (3) High versatility: It is applicable to various types of partial differential equations such as convection equation, heat conduction equation, Poisson equation, and reaction equation, and supports solving both forward and inverse problems. It has a wide range of applications. Attached Figure Description
[0039] Figure 1: Comparison of learning paradigms and visualizations of traditional PINNs and the method of this invention. (a) shows the local differential constraint paradigm of traditional PINNs; (b) and (c) show the visualizations of the solutions of traditional PINNs after 2000 and 5000 training rounds, respectively; (d) shows the global pattern regularization paradigm of this invention; and (e) and (f) show the visualizations of the solutions of the method of this invention after the corresponding training rounds.
[0040] Figure 2: Schematic diagram of spatiotemporal discretization. (a) Discretization of the unbounded spatiotemporal domain (taking the convection equation as an example), (b) Discretization of the bounded spatial domain (taking the two-dimensional Poisson equation as an example).
[0041] Figure 3: Visual comparison of solution results of different methods on complex partial differential equations.
[0042] Figure 4: Error-iteration number curves (logarithmic scale) of different methods on complex partial differential equations. Detailed Implementation
[0043] The method described in this invention is implemented based on the following experimental environment: the operating system is Windows 10, the programming language is Python 3.8.3, the deep learning framework is PyTorch 1.7.0, and an NVIDIA GTX 3080 (16G) GPU (CUDA 11.4) is used for accelerated computation. The implementation process is explained in detail below using convection equations (linear) and reaction equations (nonlinear) as examples:
[0044] 1. Implementation of linear partial differential equations (convection equations)
[0045] Step 1: Problem Definition and Discretization
[0046] The convection equations are in the following form: , where x∈[0, 2] ],t∈[0,1],β=70(high difficulty coefficient;boundary condition: u(0,t) = u(2 ,t) (periodic boundary); initial condition: u(x,0) = sin(x).
[0047] Discretize the spatiotemporal domain: Discretize the time domain t∈[0,1] into =100 points, time step τ=0.01; spatial domain x∈[0, 2] Discretized into =200 points, spatial step size h=0.0314.
[0048] Step 2: Calculation of Long-Range Independence Matrix
[0049] Discretizing the convection equations using the finite difference method yields: ,in The discretized relation is represented in matrix-vector form. Where A is The correlation matrix. Pre-calculate the matrix exponentiation. (n=10,20,……100), used to construct the global regularization loss.
[0050] Step 3: Neural Network Training
[0051] A fully connected neural network is constructed, containing 4 hidden layers, each with 128 neurons, and using ReLU as the activation function. The total loss function is: , where θ is the set of learnable parameters of the Physical Information Neural Network (PINNs), which includes all the weights and biases to be optimized in the network.
[0052] For boundary condition loss, MSE loss is adopted;
[0053] The initial condition loss is calculated using MSE loss.
[0054] The loss is the global pattern regularization loss, in the form of: ;
[0055] The weight parameter is set to =1.0, =1.0.
[0056] Using the Adam optimizer, with an initial learning rate of 0.001, train for 20,000 epochs until the loss converges.
[0057] Step 4: Online Solving and Verification
[0058] The trained model is applied to solve the convection equations, taking arbitrary spatiotemporal coordinates (x,t) as input and outputting the corresponding solution u(x,t). Compared with traditional methods such as PINNs, NTK, and MultiAdam, this method has relatively... The error was 9.49% and the training time was 80.4 seconds, which is significantly better than the traditional method (the lowest relative error was 85.76% and the longest training time was 913.9 seconds).
[0059] 2. Implementation of Nonlinear Partial Differential Equations (Reaction Equations)
[0060] Step 1: Problem Definition and Discretization
[0061] The reaction equation is in the following form: , where x∈[0, 2] ], t∈[0,1], =20 (High difficulty level); Boundary condition: (u(0,t) = u(2) ,t) (periodic boundary); initial condition: u(x,0) = .
[0062] The spatiotemporal discretization parameters are consistent with the convection equations: =100, =200.
[0063] Step 2: Definition of long-range nonlinear correlation function
[0064] Discretizing the reaction equation using the finite difference method yields: ,in It is a nonlinear function. Define the long-range correlation function. ( (representing p-fold composition of g), used to construct the global regularization loss.
[0065] Step 3: Neural Network Training
[0066] Using the same neural network structure as the convection equation, the total loss function is: The weight parameters are set to... =1.0, =1.0. The training parameters are consistent with the convection equations, and the training is conducted for 20,000 rounds.
[0067] Step 4: Online Solving and Verification
[0068] This method is relatively... in terms of reaction equations The error was 4.24%, and the training time was 172.9 seconds, while the traditional method had the lowest relative error of 58.07% and the longest training time of 739.5 seconds, which fully demonstrates the advantages of this method in solving highly difficult nonlinear problems.
Claims
1. A method for optimizing physical information neural networks based on global pattern regularization, characterized in that, Includes the following steps: (1) For the target partial differential equation, its spatiotemporal domain is discretized, and the long-range correlation between discrete points in the spatiotemporal domain is derived based on the finite difference method. The long-range correlation is independent of the specific boundary / initial conditions. (2) Based on the long-range correlation, a global pattern regularization loss function is constructed to replace the local differential operator residual loss in the traditional physical information neural network; (3) Using "global pattern regularization loss + boundary condition loss + initial condition loss" as the total loss function, train the neural network to obtain the optimized physical information neural network model. (4) Use the trained model to solve the target partial differential equation to achieve rapid propagation and prediction of the solution in the spatiotemporal domain.
2. The method according to claim 1, characterized in that, The derivation method of the long-range correlation described in step (1) is determined according to the type of partial differential equation: • For linear partial differential equations, the matrix mapping relationship between discrete solution vectors is obtained by finite difference discretization, and then the long-range correlation between any two discrete layers is derived, which can be expressed as a matrix power form or a system of linear equations. For nonlinear partial differential equations, the nonlinear recursive relation of the solution is obtained by finite difference discretization, and then the long-range nonlinear correlation between any two discrete layers is derived, which is expressed as a composite form of nonlinear functions.
3. The method according to claim 2, characterized in that, The linear partial differential equations include the convection equation, the heat conduction equation, and the two-dimensional Poisson equation, and their corresponding long-range correlations are as follows: • Convection equations, unbounded domain: u p =A p-q u q Where A is the correlation matrix and p>q is the time layer index; • Two-dimensional Poisson equation, bounded domain: Au col =h 2 f col +Bu bc Where A and B are coefficient matrices, u col Let u′ be the internal point solution vector. bc Let f be the boundary point solution vector. col Let h be the source term vector and h be the spatial step size.
4. The method according to claim 2, characterized in that, The nonlinear partial differential equations include reaction equations, whose long-range correlation is u. p =G(u q ), where G is a composite function of nonlinear recursive functions, and p>q is the time layer index.
5. The method according to claim 1, characterized in that, The form of the global pattern regularization loss function described in step (2) is determined according to the type of partial differential equation: Linear equations in unbounded domains: Among them u0(θ), u p (θ) represent the solution vectors of the initial time layer and the p-th time layer predicted by the neural network, respectively. p The power of the pre-calculated long-range correlation matrix; • Bounded domain linear equations: Where u col (θ), u bc (θ) are the predicted solution vectors for the interior points and the boundary points, respectively; • Nonlinear equations: Among them G p This represents the p-th composite of a nonlinear function G.
6. The method according to claim 1, characterized in that, The neural network described in step (3) is a fully connected neural network, trained using the Adam optimizer, and the boundary condition loss and initial condition loss in the total loss function both use MSE loss.
7. The method according to claim 1, characterized in that, The correlation matrices (such as A and B) or nonlinear functions of the long-range correlation mentioned in step (1) are pre-calculated in the offline stage, and the matrix powers are calculated and stored in advance to reduce the computational overhead in the training stage.
8. The method according to claim 1, characterized in that, The method is applicable to solving both forward and inverse problems of linear and nonlinear partial differential equations, including convection equations, heat conduction equations, Poisson equations, and reaction equations.
9. The method according to claim 1, characterized in that, The method constructs long-range connections through global pattern regularization, accelerating the propagation of the solution from the boundary / initial region into the spatiotemporal domain.