Method for solving partial differential equation through physical neural network of time evolution residual
By decomposing large-scale spatiotemporal problems into sub-periods and introducing exponentially decaying time weights and backward compatibility losses, the local optimality and discontinuity problems of PINN in solving complex PDEs are solved, achieving a more efficient and accurate solution.
Patent Information
- Application Number
- CN202510783237.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-12
- Publication Date
- 2025-09-16
Smart Images
Figure SMS_1 
Figure SMS_2 
Figure SMS_5
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of neural network solution of partial differential equations, and in particular relates to a time evolution residual training method. Background Art
[0002] Partial differential equations (PDEs), as an important tool for describing physical phenomena, occupy a central position in science and engineering. Whether it is the Navier-Stokes equations in fluid mechanics, the heat equation in heat conduction, or the wave equation in elasticity, PDEs, with their powerful mathematical expression capabilities, accurately depict various complex phenomena and processes in the physical world. However, solving these PDEs is not an easy task, especially when the problem involves complex geometries, multi-physics coupling, and nonlinear effects. Traditional numerical methods often face huge challenges, are computationally expensive, and are extremely sensitive to initial conditions and parameter settings.
[0003] The emergence of physical-informed neural networks (PINNs) has brought new hope for solving this problem. PINNs cleverly embed physical laws into the neural network's loss function, enabling the network to not only learn data features but also follow physical laws during training. This approach, without requiring large amounts of data, can process complex differential equations through automatic differentiation techniques, significantly expanding the application boundaries of deep learning in scientific computing. In recent years, PINNs have made significant progress in multiple fields. In fluid mechanics, PINNs have successfully solved the Navier-Stokes equations, enabling accurate predictions of fluid velocity, pressure, and temperature fields, providing new tools for turbulence simulation and flow control. In heat transfer problems, PINNs can accurately simulate heat distribution and transfer processes, even in data-scarce situations, providing physically plausible solutions, demonstrating their enormous potential in thermal process modeling and design optimization. Furthermore, PINNs are increasingly being applied in fields such as structural mechanics, electromagnetics, and biomedicine, providing new insights and methods for solving complex engineering and scientific problems.
[0004] However, PINNs still face many challenges when dealing with complex, nonlinear, and unsteady PDEs. For example, in high-Reynolds number flow problems, PINNs are prone to falling into local optima, resulting in inaccurate solutions. To address this issue, researchers have proposed various improvement strategies, such as introducing new loss function terms, optimizing network architecture, and employing adaptive training algorithms. PINNs also have difficulties dealing with convection, reaction, and diffusion equations with nontrivial coefficients. To address these issues, researchers have proposed methods such as curriculum regularization and sequence-to-sequence learning, which have effectively improved the performance and applicability of PINNs. Significant progress has been made in the use of PINNs (physical information neural networks) to solve complex, nonlinear, and unsteady partial differential equations (PDEs). Since their introduction by Raissi et al., this method has provided a novel approach to solving PDEs by incorporating physical laws into the loss function of neural networks. However, PINNs have exposed many shortcomings when dealing with complex problems. To this end, Karniadakis's team proposed a conservative physical information neural network (cPINN). By decomposing the computational domain and enforcing flux continuity, they improved the accuracy and efficiency of PINN for nonlinear conservation law problems. Krishnapriyan et al., meanwhile, conducted an in-depth analysis of PINN failure modes, finding that it has difficulties handling convection, reaction, and diffusion equations with nontrivial coefficients. They proposed improvement strategies such as curriculum regularization and sequence-to-sequence learning. These studies have collectively advanced the accuracy and efficiency of PINN in solving complex, nonlinear, and unsteady PDEs, expanding its application across diverse scientific fields. In fluid mechanics, Jagtap et al. focused on the limitations of PINN for problems such as high-Reynolds number flows. By introducing new loss functions and optimization algorithms, they improved the applicability of PINN in this field. In materials science, Sahli Costabal et al. applied PINN to multiphysics problems, successfully predicting the mechanical behavior of materials under complex loads and providing new tools for material design and optimization. In astronomy and space science, PINN has also been used to analyze cosmological simulation data and solar wind plasma behavior, demonstrating its potential for processing large-scale scientific data. Wang et al. also focused on the computational efficiency of PINN. By introducing Taylor-mode automatic differentiation (Taylor-mode AD) and data parallelization strategies, they significantly improved the computational efficiency of PINN in processing high-order derivatives and large-scale problems.
[0005] The core concept of the proposed method is to decompose large-scale spatiotemporal problems into multiple sub-periods and adopt a time-domain segmented training strategy to reduce the complexity of single training runs and ensure the spatiotemporal continuity of the solution. Furthermore, an exponentially decaying time weight function is introduced to dynamically adjust the contribution of different time layers to the loss, allowing the model to focus on early time layers before gradually incorporating later details. Summary of the Invention
[0006] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is as follows:
[0007] A method for solving partial differential equations using a physical neural network with time-evolving residuals includes the following steps:
[0008] A sequential activation time function in the physical time domain is designed, which activates the physical domain information of future time in sequence through the backward progress of the training process.
[0009] Introducing backward compatibility loss. This loss term requires that all losses at the intersection of past and future data be consistent, that is, the data at future activation points must be updated based on the data at previous activation points;
[0010] Under the modified conditions after the loss of the time evolution coupled physical conditions, it should be ensured that the model loss function is adaptively regulated to meet the solution of the time evolution physical partial differential equations; the partial differential equations are solved through a physical information neural network based on time evolution residuals and regional training.
[0011] The beneficial effects of the present invention are as follows: the present invention provides a method for solving partial differential equations using a time-evolved residual and a physical information neural network trained in different regions, and proposes a physical information neural network structure that is gradually trained in the order of time evolution, which is weighted by dynamic time weights to simultaneously meet physical constraints and data fitting requirements, thereby comprehensively improving the accuracy and convergence speed of the model when dealing with complex physical problems. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings used in the embodiments.
[0013] Figure 1 It is a flowchart of the steps of the method for solving partial differential equations using a time-evolved residual and a physical information neural network trained by region in the embodiment;
[0014] Figure 2 1 and 2 are the predicted results (a) and absolute error (b) of the PINN of the Allen-Cahn equation in the embodiment.
[0015] Figure 3 Graph (a) and graph (b) of the absolute error of the Allen-Cahn equation model in the examples.
[0016] Figure 4 Graph (a) and graph (b) of absolute error of the prediction results of the present PINN model for the convection equation in the embodiment.
[0017] Figure 5 1 and 2 are the prediction result diagram (a) and absolute error diagram (b) of this model for the convection equation in the embodiment. DETAILED DESCRIPTION
[0018] To make the technical solution of the present invention clearer, the present invention will be clearly and completely explained below with reference to the accompanying drawings of the embodiments of the invention. The embodiments described only use a part of the embodiments, not all of the embodiments. The present invention is specifically implemented according to the following steps:
[0019] (1) To solve the problems of gradient vanishing, numerical instability and local optimum encountered when directly training PINN in large-scale spatiotemporal problems, we divide the overall time interval [t0, t1] into K sub-intervals [T k ,T k+1 ](k=0,1,…,K-1).
[0020] (2) The prediction results of the previous sub-period are used as the initial conditions for the next period to ensure the continuity of the overall solution in time and space. For each sub-period, the initial conditions are set as follows:
[0021]
[0022] Where u0(x) represents the initial condition given in the problem, u prev (x,T k ) is the end time of the previous sub-period T k Decomposing the global problem into local problems reduces the complexity of a single training run and utilizes sufficient fitting within the local time period to transfer the state segment by segment.
[0023] (3) To address the uneven contribution of different time layers to the overall loss during training, an exponentially decaying time weight function ω(t;n) is designed. The specific form is as follows:
[0024]
[0025] Where t is the current physical time, T Start and T end are the start and end time of the current sub-period, n is the number of iterations of the current training, N is the total number of iterations in the sub-period, x] + =max{x,0} is the positive function, and λ is the hyperparameter of the decay rate.
[0026] (4) Furthermore, when When , the weight ω(t;n) = 1, there is no additional attenuation, and the current physical time point of the current iteration number is fitted according to the normal weight; when When , the loss of later physical time points will be attenuated, which helps to stabilize the early fitting first and then gradually introduce later data information. However, as the number of iterations n continues to increase, the physical time point of the attenuated weight will gradually shift back, that is, in the later stages of the iteration, the model tends to learn the behavior of the PDE system in the entire domain.
[0027] (5) During the time-domain segmented training process, since the training update of each sub-period may interfere with the information learned in the previous period, in order to prevent "catastrophic forgetting", a backward compatibility loss is introduced. The specific form is as follows:
[0028]
[0029] Among them L back For backward compatibility loss, X back is the set of saved sampling points, Y back is the corresponding prediction result. By introducing the backward compatibility loss, it can be regarded as imposing regularization constraints on the model parameters, ensuring that while optimizing the data of the new sub-period, it will not significantly deviate from the previously fitted solution, thereby effectively reducing the spatiotemporal discontinuities that may be caused in the subsequent update process.
[0030] (6) In order to simultaneously meet the physical constraints and data fitting requirements, the comprehensive loss function integrates various objectives and weights the boundary condition loss and PDE residual loss through dynamic time weights. Specifically, the total loss function is designed as follows:
[0031] L total =L ic +ω(t;n)L bc +ω(t;n)L res +L back
[0032]
[0033] Among them L ic is the initial condition loss, L bc is the boundary condition loss, L res is the PDE residual loss, L back is the backward compatibility loss. is the boundary condition operator, is the operator of the PDE being sought.
[0034] (7) Decompose the large-scale spatiotemporal problem into multiple sub-periods and adopt a time-domain segmented training strategy to reduce the complexity of single training and ensure the spatiotemporal continuity of the solution. At the same time, introduce an exponentially decaying time weight function to dynamically adjust the contribution of different time layers to the loss, so that the model first focuses on the early time layers and then gradually introduces later details.
[0035] (8) As mentioned above, the space-time region optimized during the neural network solution is gradually stepped in the order of time evolution, which is more in line with the internal physical laws of PDE.
[0036] The present invention provides two embodiments, namely, solving the Allen-Cahn equation and solving the convection equation under high convection coefficient.
[0037] Example 1: Solving the Allen-Cahn equation, the specific scheme is as follows:
[0038] The Allen-Cahn equation is a classic nonlinear partial differential equation widely used in fields such as materials science and phase transition dynamics, describing the evolution of matter between different phases. Its mathematical form is:
[0039]
[0040] Where u represents the phase field variable of the substance, ν is the diffusion coefficient and is taken as 0.0001, 5u(u 2 -1) is a nonlinear term. The initial conditions are:
[0041] u(x,0)=x 2 cos(πx)
[0042] The boundary conditions are periodic:
[0043] u(-1,t)=u(1,t)
[0044] u x (-1,t)=u x (1,t)
[0045] The total number of training iterations was 50,000, with an initial learning rate of 1e-3. The StepLR learning rate scheduler was used, multiplying the learning rate by 0.9 every 5,000 iterations. Uniform sampling was used, and the results were compared with the predictions of a standard physical information neural network. The proposed method is used to solve the Allen-Cahn prediction graph, exact solution graph, and absolute error graph, as shown in the figure.
[0046] Example 2: Solving the convection equation under high convection coefficient. The specific solution is as follows:
[0047] The convection equation is a basic partial differential equation in fluid mechanics and transport phenomena that describes the propagation of scalar fields (such as concentration, temperature, etc.) with fluid motion. Its mathematical form is:
[0048]
[0049] Where c is the convection coefficient and is set to 60. The initial conditions are:
[0050] u(x,0)=sin(x)
[0051] Use periodic boundary conditions:
[0052] u(0,t)=u(2π,t)
[0053] The training setting is the same as that in Example 1. The result prediction diagram, accurate solution diagram, absolute error diagram, etc. for solving the convection equation of the present invention are shown in the figure. The above is only a preferred embodiment of the present invention and is not used to limit the present invention. For those skilled in the art, the present invention can have various modifications and changes.
Claims
1. A method for solving partial differential equations using a physical neural network with time-evolving residuals, characterized in that: The steps include: A temporal function is designed that is activated sequentially in the physical time domain. Within each sub-period, the model focuses on earlier time layers in the early stages of training, and gradually introduces later details that are more difficult to fit. Divide the overall time interval into several sub-periods. In regional training, the model is optimized independently in each sub-period, making local problems easier to solve. Introducing a backward compatibility loss, which forces the model to keep the predictions of sampling points in the previous sub-period consistent with the saved prediction results during the current update, thereby achieving effective transfer of knowledge across time periods; Under the modified conditions after the loss of the time-evolution coupled physical conditions, it should be ensured that the model loss function is adaptively regulated to meet the solution of the time-evolution physical partial differential equations; The partial differential equations are solved through a physical information neural network based on time-evolving residuals.
2. The method for solving partial differential equations using a physical neural network with time-evolving residuals according to claim 1, characterized in that: The specific form of the exponential decay time weight function is as follows: Where t is the current physical time, T Start and T end are the start and end time of the current sub-period, n is the number of iterations of the current training, N is the total number of iterations in the sub-period, [x] + =max{x,0} is the positive function, and λ is the hyperparameter of the decay rate. This function realizes differentiated weighted control of loss terms in different time periods through the dynamic association between the iterative process and the time position.
3. The method for solving partial differential equations using a physical neural network with time-evolving residuals according to claim 1, characterized in that: The specific form of the regional training is as follows: the overall time interval [t0, t1] is divided into K sub-periods [T k ,T k+1 ](k=0,1,…,K-1), for each sub-period, the initial conditions are set as follows: where u0(x) represents the initial condition provided by the partial differential equation in the given physical problem, and u prev (x,T k ) is the end time of the previous sub-period T k prediction results.
4. The method for solving partial differential equations using a physical neural network with time-evolving residuals according to claim 1, characterized in that: The backward compatibility loss is as follows: By introducing such a backward compatibility loss, it can be regarded as imposing regularization constraints on the network model parameters, so that the interface conditions of the segmented solution domain meet the consistency of the physical problem, thereby ensuring that the problem solved after segmentation is consistent with the original problem and conforms to the law of time evolution.
5. The method for solving partial differential equations using a physical neural network with time-evolving residuals according to claim 1, characterized in that: The boundary condition loss and PDE residual loss are weighted by dynamic time weights, and the complex unsteady problem solved by the physical neural network is sequentially activated in the time and space domain to make it conform to the time and space forward flow characteristics of solving physical problems, thereby achieving effective solution to the complex unsteady problem and reducing the number of solution iterations.