Novel fractional order physical information convolutional recurrent neural network solver

Abstract

The invention relates to the field of numerical calculation of fractional order partial differential equations (groups), and discloses a novel fractional order physical information convolutional recurrent neural network solver, which comprises the following steps of: 1, determining the fractional order partial differential equations (groups) and initial conditions satisfied by a calculation region, 2, a neural network is constructed according to a self-defined coding-convolution LSTM-decoding layer structure, the input of the neural network is an initial condition, and the output of the neural network is a prediction solution of a fractional order partial differential equation (group); 3, calculating the loss of the control equation to obtain a loss function; and 4, obtaining an optimal loss function weight and parameters by using an optimization algorithm, obtaining a solved fractional order partial differential equation (group) solving model, and solving the fractional order partial differential equation (group). Compared with a fractional order physical information neural network (fPINNs), the fractional order partial differential equation (group) solving method based on the physical information convolutional recurrent neural network has remarkable advantages in extrapolation and expansibility, and the problems that in practical application, the fPINNs network is large in parameter quantity and poor in extrapolation and generalization ability are solved.

Description

Technical Field

[0001] The present invention relates to the field of numerical calculation of fractional-order partial differential equations (groups), and in particular to a novel fractional-order physical information convolutional recurrent neural network solver. Background Art

[0002] The study of fractional-order partial differential equations (groups) has a long history and has developed rapidly in the past decade. By introducing fractional-order derivatives, the dependence of the system on past states can be captured, thereby more accurately simulating dynamic behaviors with historical memory characteristics. Analytical solutions to fractional-order partial differential equations (groups) are usually difficult to find, so the application of numerical methods is very important in solving such equations. Fractional-order partial differential equations have a wide range of applications in fluid mechanics, nonlinear dynamics, and materials science. For example, fractional-order partial differential equations play a key role in solute transport in porous media and anomalous diffusion in inhomogeneous media. Finite difference methods, finite element methods, and spectral methods are usually used to solve fractional-order partial differential equations (groups), but these methods have great limitations, need to verify the stability of the format, and require a lot of computing resources.

[0003] With the rapid development of deep learning, using neural networks to solve partial differential equations has become a research hotspot. Some researchers have proposed fractional-order physical information neural networks (fPI NNs), which are fully connected neural networks that add the equations to be solved as residuals to the loss function, update the model parameters through the optimizer, and predict the solutions to the equations. However, in practical applications, the number of parameters of fPI NNs networks is large, and the extrapolation and generalization capabilities are poor. We apply a new fractional-order physical information convolutional recurrent neural network solver to solve fractional-order partial differential equations (groups). Experimental results show that the new network has better extrapolation and stronger generalization capabilities. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention provides a novel fractional-order physical information convolutional recurrent neural network solver, which solves the problems of large number of parameters, poor extrapolation and generalization capabilities of fPI NNs networks in practical applications.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: a novel fractional-order physical information convolutional recurrent neural network solver, comprising the following steps:

[0006] Step 1: Determine the fractional-order partial differential equation (group) and initial conditions satisfied by the computational domain;

[0007] Step 2: Build a neural network based on the customized encoding-convolutional LSTM-decoding structure, input the initial conditions, output the predicted solution of the fractional-order partial differential equation (group) through the neural network, and establish a solution model for the fractional partial differential equation (group);

[0008] Step 3: Calculate the control equation loss and obtain the loss function;

[0009] Step 4: Use the optimization algorithm to obtain the optimal loss function weights and neural network parameters, and obtain the solution model of the fractional-order partial differential equation (group).

[0010] Preferably, the fractional-order partial differential equation (group) in step 1 is defined as follows:

[0011]

[0012] in, is the α-order Caputo fractional derivative with respect to u, represents the gradient of u with respect to the spatial variable x, represents the Laplacian operator of u with respect to the spatial variable x, represents the dot product of the gradient of u and u itself. The function F represents a nonlinear term that depends on u and its various derivatives, and also on the parameter λ, x = [x, y] T is the space-time coordinate, u=[u,ν] T is the predicted solution of the equation (group of equations), and f represents the source term of the equation.

[0013] Preferably, the custom neural network in step 2 is f θ (x, t), the encoding block contains three convolutional layers, the output channels are 8, 32, and 128 respectively, using a 4×4 convolution kernel, a convolution step of 2, and circular padding. The convolutional LSTM structure is defined as follows:

[0014]

[0015] Among them, the input of the network is X t ,{h t ,C t} are the hidden state and cell state updated at time t, respectively. The core components of the convolutional LSTM include the input gate i t , Forget gate t , unit state C t and output gate o t ,* represents convolution operation, ⊙ represents Hadamard product, {W i ,W f ,W c ,W o} is the weight parameter of the corresponding convolution kernel, {bi ,b f ,b c ,b o} is the bias vector and σ represents the sigmoid activation function.

[0016] Preferably, the equation residual in step 3 is specifically defined as follows:

[0017] .

[0018] in Using the L1 format approximation, a 5×5 convolution kernel is constructed using the peridynamic differential operator to calculate the spatial integer-order derivatives of u.

[0019] Preferably, the control equation loss in step 3 is Has the following form:

[0020] Where n and m represent the number of grid points in the x and y directions respectively, T refers to the total number of time steps, and ||·||2 refers to the l2 norm. It refers to the point in space (x i ,y j ) and time point t k The residual of the fractional-order partial differential equation (system).

[0021] Preferably, the optimization algorithm in step 4 is an adaptive optimization algorithm, including an Adam optimization algorithm or an RMSProp optimization algorithm, which dynamically adjusts the weight parameters of the neural network by updating the learning rate to ensure rapid convergence and stability of the model.

[0022] Preferably, the input gate, forget gate, unit state and output gate of the convolutional LSTM structure are nonlinearly activated using sigmoid and tanh functions respectively, and are operated on the spatiotemporal sequence through the convolution kernel to better capture the spatiotemporal correlation of the fractional-order differential equation.

[0023] The present invention provides a novel fractional-order physical information convolutional recurrent neural network solver, which has the following beneficial effects:

[0024] 1. The method for solving fractional-order partial differential equations (groups) based on physical information convolutional recurrent neural network proposed in the present invention has significant advantages in extrapolation and scalability compared with fractional-order physical information neural network.

[0025] 2. A new type of fractional-order physical information convolutional recurrent neural network proposed in the present invention has more advantages than traditional finite difference method, finite element method and spectral method in dealing with some problems. For example, it does not require gridding of the computational domain, thus avoiding the difficulty of complex gridding in high-dimensional space, and reducing the errors and instability caused by the difference format. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 Flowchart for solving the problem using a novel fractional-order physics-informed convolutional recurrent neural network solver;

[0027] Figure 2 Schematic diagram of a novel fractional-order physical information convolutional recurrent neural network solver;

[0028] Figure 3 A network diagram of the convolutional LSTM structure.

[0029] Figure 4 The results of Example 1 are as follows;

[0030] Figure 5 This is the result of Example 2. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the specification of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] Embodiment 1:

[0033] Please see attached Figure 1 -Attached Figure 5 , an embodiment of the present invention provides a novel fractional-order physical information convolutional recurrent neural network solver, comprising the following steps:

[0034] Step 1: Determine the fractional-order partial differential equation (group) and initial conditions satisfied by the computational domain;

[0035] Step 2: Build a neural network based on the custom encoding-convolutional LSTM-decoding structure, input the initial conditions, output the predicted solution of the fractional-order partial differential equation (group) through the neural network, and establish a solution model for the fractional-order partial differential equation (group);

[0036] Step 3: Calculate the control equation loss and obtain the loss function;

[0037] Step 4: Use the optimization algorithm to obtain the optimal loss function weights and neural network parameters, and obtain the solution model of the fractional-order partial differential equation (group).

[0038] Specifically, first, step 1 determines the fractional-order partial differential equation (group) and initial conditions satisfied in the computational domain; then, step 2 constructs a neural network according to a custom encoding-convolutional LSTM-decoding structure, the encoding part extracts the spatial features of the input data through multi-layer convolution, the convolutional LSTM module is used to capture the long-term dependence of spatiotemporal information, the decoding part restores the predicted solution, and after inputting the initial conditions of the system, forward propagation is performed through the neural network to output the predicted solution of the fractional-order partial differential equation (group), and establish a solution model for the fractional-order partial differential equation (group); then, step 3 discretizes the time fractional-order derivatives using a finite difference format, and constructs the convolution kernel of the spatial derivative using a peridynamic differential operator, thereby obtaining the residual of the control equation, and defining the loss function of the control equation, the loss function form is the average value of the square of the residual, so as to ensure that the network can effectively capture the physical constraints of the equation; finally, step 4 uses an adaptive optimization algorithm (such as Adam or RMSProp) to adjust the network parameters and the weight of the loss function, and uses the gradient descent method to continuously optimize the network until the loss function converges, and finally obtains the optimal solution model for the fractional-order partial differential equation (group).

[0039] The custom neural network in step 2 is f θ (x, t), the encoding block contains three convolutional layers, the output channels are 8, 32, and 128 respectively, using a 4×4 convolution kernel, a convolution step of 2, and circular padding. The convolutional LSTM structure is defined as follows:

[0040]

[0041] Among them, the input of the network is X t ,{h t ,C t} are the hidden state and cell state updated at time t, respectively. The core components of the convolutional LSTM include the input gate i t , Forget gate t , unit state C t and output gate o t ,* represents convolution operation, ⊙ represents Hadamard product, {W i ,W f ,W c ,W o} is the weight parameter of the corresponding convolution kernel, {b i ,b f ,b c ,b o} is the bias vector and σ represents the sigmoid activation function.

[0042] Specifically, the custom neural network adopts the encoding-convolutional LSTM-decoding structure, in which the encoding block consists of three convolutional layers, the number of output channels of the convolutional layers is 8, 32 and 128 respectively, the convolution kernel size is 4×4, the convolution step is 2, and the ring filling method is used to keep the input and output dimensions consistent. In the convolutional LSTM structure, the input of the network includes the hidden state and cell state updated at each time step, and the input data is processed by convolution operation. The core of the convolutional LSTM consists of an input gate, a forget gate, a cell state and an output gate. Each gated unit is used to control the input, forgetting and output of information. * represents convolution operation, represents Hadamard product, and the weight parameters of the convolution kernel and the bias vector are used to adjust the learning ability of the neural network. The gated unit performs nonlinear processing through the sigmoid activation function to control the flow of information. Through this structure, the network can effectively capture complex dependencies in spatiotemporal sequences.

[0043] The control equation loss of step 3 Has the following form:

[0044] Where n and m represent the number of grid points in the x and y directions respectively, T refers to the total number of time steps, and ||·||2 refers to the l2 norm. It refers to the point in space (x i ,y j ) and time point t k The residuals of the governing equations.

[0045] Specifically, the loss function is used to measure the deviation between the predicted solution of the neural network and the true physical solution. Its specific form combines the error calculation in space and time. The loss function takes a weighted average of all the calculation points in the entire spatial grid, comprehensively considers the error in the spatial dimension, and accumulates the prediction results of all time steps to ensure the overall accuracy of time series prediction. In addition, the loss function uses the l2 norm in the mathematical formula to measure the difference between the predicted solution and the true solution at different time steps, ensuring that the prediction error of the model in space and time is minimized, thereby improving the accuracy of solving fractional partial differential equations. By calculating this loss function, the model can be guided to optimize the parameters so that the neural network gradually approaches the true physical solution.

[0046] This embodiment takes the following fractional-order Burgers' equation (group) as an example:

[0047]

[0048] Where x = [x, y] T is the space-time coordinate, u=[u,ν] Tis the solution of the equation (group), parameter λ = 0.01, f1, f2 represent the source terms of the equation (group), and the subscript represents the partial derivative with respect to the variable.

[0049] The calculation area in step 1 is (x, y)∈[0,1]×[0,1], and the initial condition is set to u0=0.

[0050] The input gate, forget gate, unit state and output gate of the convolutional LSTM structure are nonlinearly activated using sigmoid and tanh functions respectively, and are operated on the spatiotemporal sequence through the convolution kernel to better capture the spatiotemporal correlation of the fractional-order differential equation.

[0051] The equation residual in step 3 is specifically defined as follows:

[0052]

[0053] . Discrete-time fractional derivatives using L1 format in l ≥ 0, τ represents the time step. The spatial integer derivative uses the peridynamic differential operator to construct the convolution kernel and calculate u x and v x The convolution kernel used is

[0054] Calculate u y and v y The convolution kernel used is

[0055]

[0056] Calculate u xx +u yy and v xx +v yy The convolution kernel used is

[0057]

[0058] The control equation loss of step 3 Has the following form:

[0059] The spatial region [0,1]×[0,1] is evenly divided into 128×128, and training is performed at 10 uniformly discrete time points t∈[0,1] with a time step of 0.1. Then n=m=128,T=10 in the loss function.

[0060] The optimization algorithm in step 4 is selected as the Adam optimization algorithm. The learning rate is multiplied by 0.97 after every 100 epochs. This learning rate decay strategy helps the model converge quickly in the early stage of training. In the later stage of training, the model parameters are refined by reducing the learning rate to improve the generalization ability of the model.

[0061] In step 4, the time step τ is set to 0.1, the training phase t∈[0,1], and the extrapolation phase t∈[1,1.5]. The initial learning rate is set to 0.001. In order to verify the effectiveness of the model, it is assumed that the equation (group) has an exact solution u(x,y,t)=t 1.5 sin(πx)sin(πy), The corresponding

[0062]

[0063]

[0064] The number of iterations is set to 2000, and the fPINNs model is used for comparison. The fPINNs model uses the Adam optimizer, the learning rate is set to 0.001, and the number of training times is 20000. For the two-dimensional fractional-order Burgers equation, the performance of the new fractional-order physical information convolutional recurrent neural network solver and fPINNs in terms of solution accuracy and extrapolation is compared. Three representative time points are selected: the training stage (t = 0.5, 1) and the extrapolation stage (t = 1.5). The test results are shown in Figure 4 As shown in the figure, “Err(ours)” and “Err(fPINNs)” refer to the difference between the predicted values ​​and the true values ​​of the two methods in the entire domain. Figure 4 It can be concluded that the new fractional-order physical information convolutional recurrent neural network has significant advantages over fPINNs in extrapolation and scalability.

[0065] Example 2: This example takes the following fractional-order convection-diffusion equation as an example

[0066]

[0067] Parameters λ=0.01, γ=0.1.

[0068] step:

[0069] The calculation area in step 1 is (x, y)∈[0,1]×[0,1], and the initial condition is set to u0=0.

[0070] Build a neural network model: Use the encoding-convolutional LSTM-decoding structure in step 2. The encoding layer uses 3 convolutional layers. The convolutional LSTM handles spatiotemporal dependencies. The decoding layer restores the output size to the same as the input.

[0071] The equation residual in step 3 is specifically defined as follows:

[0072] Discrete-time fractional derivatives using L1 format in τ represents the time step. The spatial integer derivatives use the peridynamic differential operator to construct the convolution kernel and calculate u x The convolution kernel used is

[0073] Calculate u y The convolution kernel used is

[0074]

[0075] Calculate u xx +u yy The convolution kernel used is

[0076]

[0077] The control equation loss of step 3 Has the following form:

[0078] The spatial region [0,1]×[0,1] is evenly divided into 128×128, and training is performed at 10 uniformly discrete time points t∈[0,1] with a time step of 0.1. Then n=m=128,T=10 in the loss function.

[0079] The optimization algorithm in step 4 is selected as the Adam optimization algorithm. The learning rate is multiplied by 0.97 after every 100 epochs. This learning rate decay strategy helps the model converge quickly in the early stage of training. In the later stage of training, the model parameters are refined by reducing the learning rate to improve the generalization ability of the model.

[0080] In step 4, the time step τ is set to 0.1, the training phase t∈[0,1], and the extrapolation phase t∈[1,1.5]. The initial learning rate is set to 0.001. In order to verify the effectiveness of the model, it is assumed that the equation (group) has an exact solution u(x,y,t)=t 1.5 sin(πx)sin(πy), then the corresponding

[0081]

[0082] The number of iterations is set to 2000, and the fPINNs model is used for comparison. The fPINNs model uses the Adam optimizer, the learning rate is set to 0.001, and the number of training times is 20000. For the fractional-order reaction-diffusion equation in Example 2, the performance of the new fractional-order physical information convolutional recurrent neural network solver and fPINNs methods in terms of solution accuracy and extrapolation is compared. Three representative time points are selected: the training phase (t=0.5, 1) and the extrapolation phase (t=1.5). The test results are shown in Figure 2. Figure 5 As shown in the figure, “Err(ours)” and “Err(fPINNs)” refer to the differences between the predicted values ​​and the true values ​​of the two methods in the entire domain.

[0083] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A novel fractional-order physical information convolutional recurrent neural network solver, characterized in that: The following steps are involved: Step 1: Determine the fractional-order partial differential equation (group) and initial conditions satisfied by the computational domain; Step 2: Build a neural network based on the customized encoding-convolutional LSTM-decoding structure, input the initial conditions, output the predicted solution of the fractional-order partial differential equation (group) through the neural network, and establish a solution model for the fractional partial differential equation (group); Step 3: Calculate the control equation loss and obtain the loss function; Step 4: Use the optimization algorithm to obtain the optimal loss function weights and neural network parameters, and obtain the solution model of the fractional-order partial differential equation (group).

2. According to claim 1, a novel fractional-order physical information convolutional recurrent neural network solver is characterized in that: The fractional order partial differential equation (group) in step 1 is defined as follows: in, is the α-order Caputo fractional derivative with respect to u, represents the gradient of u with respect to the spatial variable x, represents the Laplacian operator of u with respect to the spatial variable x, represents the dot product of the gradient of u and u itself. The function F represents a nonlinear term that depends on u and its various derivatives, and also on the parameter λ, x = [x, y] T is the space-time coordinate, u=[u,ν] T is the predicted solution of the equation (group of equations), and f represents the source term of the equation.

3. The novel fractional-order physical information convolutional recurrent neural network solver according to claim 1 is characterized in that: The custom neural network in step 2 is f θ (x, t), the encoding block contains three convolutional layers, the output channels are 8, 32, and 128 respectively, using a 4×4 convolution kernel, a convolution step of 2, and circular padding. The convolutional LSTM structure is defined as follows: Among them, the input of the network is X t ,{h t ,C t } are the hidden state and cell state updated at time t, respectively. The core components of the convolutional LSTM include the input gate i t , Forget gate t , cell state C t and output gate o t ,* represents convolution operation, ⊙ represents Hadamard product, {W i ,W f ,W c ,W o } is the weight parameter of the corresponding convolution kernel, {b i ,b f ,b c ,b o } is the bias vector and σ represents the sigmoid activation function.

4. The novel fractional-order physical information convolutional recurrent neural network solver according to claim 1 is characterized in that: The equation residual in step 3 is specifically defined as follows: ， Time fractional derivative The finite difference method is used for discretization, and the peridynamic differential operator is used to construct a convolution filter for computing the spatial integer-order derivatives of u.

5. The novel fractional-order physical information convolutional recurrent neural network solver according to claim 1, characterized in that: The control equation loss of step 3 Has the following form: where n and m refer to the number of grid points in the x and y directions, T refers to the total number of time steps, and ||·||2 refers to the l2 norm. It refers to the point in space (x i ,y j ) and time point t k The residual of the fractional-order partial differential equation (system).

6. The novel fractional-order physical information convolutional recurrent neural network solver according to claim 1, characterized in that: The optimization algorithm in step 4 is an adaptive optimization algorithm, including the Adam optimization algorithm or the RMSProp optimization algorithm, which dynamically adjusts the weight parameters of the neural network by updating the learning rate to ensure the rapid convergence and stability of the model.

7. The novel fractional-order physical information convolutional recurrent neural network solver according to claim 1 is characterized in that: The input gate, forget gate, unit state and output gate of the convolutional LSTM structure are nonlinearly activated using sigmoid and tanh functions respectively, and are operated with the spatiotemporal sequence through the convolution kernel to better capture the spatiotemporal correlation of the fractional-order differential equation (group).

Images