Decomposition Fourier neural operator for parameterizing partial differential equation

Abstract

The invention provides a method for solving a parameterized partial differential equation, which comprises the following steps of: improving a parameter function into high-dimensional representation, and decomposing the high-dimensional representation into a plurality of one-dimensional vectors with the same hidden dimension; performing iterative updating on the plurality of one-dimensional vectors by using a plurality of Fourier layers which are cascaded in sequence; multiplying the plurality of iteratively updated one-dimensional vectors to obtain a plurality of matrixes of which the rank is 1, and adding the plurality of matrixes to obtain a final expression; the final representation is projected back to a low-dimensional physical understanding.

Description

Technical Field

[0001] This disclosure relates to the field of machine learning technology, and more specifically to a method for solving parameterized partial differential equations using decomposed Fourier neural operators, an electronic device, and a computer-readable storage medium. Background Technology

[0002] Partial differential equations (PDEs) play a fundamental role in various scientific and engineering disciplines due to their wide applications and importance. Efficient solution techniques are crucial for advancing scientific knowledge and technological development. Traditional numerical methods, such as the finite element method (FEM) and the finite difference method (FDM), often incur significant computational costs due to the need for repeated calculations under multiple parameter configurations. Recent advances in data-driven machine learning (ML) methods have demonstrated the potential to alleviate the computational burden associated with traditional numerical methods. These machine learning methods can learn solutions directly from data and are capable of solving PDEs in complex geometries without the need for structured meshes.

[0003] There are two main machine learning-based methods for approximating solutions to partial differential equations (PDEs): (1) directly using neural networks (NNs) to parameterize the solutions to PDEs; and (2) learning operators from the parameters and solutions of PDEs.

[0004] Operator learning aims to learn the mapping between the parameter space and the relevant solution space. By discretizing with a finite number of observations, the function space can be represented in a grid form, enabling the implementation of neural operator methods in the discretized version. Unlike Physical Information Neural Networks (PINNs), which require resolving, for new instances of parameterized partial differential equations (PDEs), obtaining the solution only requires a single forward propagation through the network, making it an efficient method for solving parameterized PDEs.

[0005] Although Fourier neural operators (FNOs) and their variants have proven to be efficient methods for solving parameterized partial differential equations (PDEs) compared to traditional numerical methods, there is still room for improvement in the computational efficiency of these methods, especially inference time. Summary of the Invention

[0006] This disclosure introduces the Decomposed Fourier Neural Operator (D-FNO), which improves the efficiency of FNO by leveraging tensor decomposition and the separability of FFT. By decomposing the high-dimensional latent representation into a series of first-order tensor products, the three-dimensional FFT is replaced by a series of one-dimensional FFTs, thereby reducing the time complexity in three-dimensional problems from O(N) to O(N). 3The time complexity of D-FNO is reduced from O(logN) to O(NlogN). Experiments demonstrate the effectiveness of D-FNO in solving the two-dimensional Poisson equation, Darcy flow equation, Navier-Stokes equation, hyperelastic equation, and three-dimensional elastic wave propagation parameter equation. Results show significant improvements in training and inference time compared to FNO and the Factorization FNO method (FFNO), while maintaining comparable accuracy. Furthermore, the impact of key hyperparameters on the performance of this method is investigated. This work highlights the potential of D-FNO in handling large-scale three-dimensional problems, emphasizing its efficiency advantages and scalability.

[0007] This disclosure provides a method for solving parameterized partial differential equations, comprising: raising the parametric function to a high-dimensional representation; decomposing the high-dimensional representation into multiple one-dimensional vectors with the same hidden dimension; iteratively updating each of the multiple one-dimensional vectors using a series of cascaded Fourier layers; multiplying the iteratively updated multiple one-dimensional vectors to obtain multiple rank-1 matrices; summing the multiple matrices to obtain a final representation; and projecting the final representation back into a low-dimensional physics solution.

[0008] In the following embodiments of this disclosure, the iterative update of each one-dimensional vector Defined as:

[0009]

[0010] Where t = 1, 2, ..., T are Fourier layer indices, d = 1, ..., D are dimension indices, p = 1, ..., P are decomposition item indices, and W... t,d It is a linear transformation of the d-th dimension, where σ represents a nonlinear activation function. Represents the Fourier transform of the function. It is its inverse transform, R t,d It is a learning core.

[0011] In the following embodiments of this disclosure, decomposing the high-dimensional representation into a plurality of one-dimensional vectors having the same hidden dimension includes using a plurality of different neural networks to decompose the high-dimensional representation into the plurality of one-dimensional vectors.

[0012] In some embodiments of this disclosure, the partial differential equations include the two-dimensional steady-state Poisson equation and Darcy flow equation, the two-dimensional unsteady-state Navier-Stokes equation, the two-dimensional hyperelastic solid equation in an irregular domain, and the three-dimensional elastic wave equation.

[0013] In some embodiments of this disclosure, the parameter function is discretized into grid data before being promoted to a high-dimensional representation.

[0014] This disclosure provides an electronic device, including:

[0015] At least one processor; and

[0016] A memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores one or more computer programs that can be executed by the at least one processor to enable the at least one processor to perform the methods described above in this disclosure.

[0018] This disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the methods described above. Attached Figure Description

[0019] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the embodiments of the present disclosure to explain the disclosure and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed description of exemplary embodiments with reference to the accompanying drawings, in which:

[0020] Figure 1 The architecture of a Fourier neural operator for two-dimensional problems is shown.

[0021] Figure 2 This is the architecture of a decompositional Fourier neural operator in a two-dimensional setting, where (a) is the overall architecture of the decompositional Fourier neural operator, (b) is the detailed update process in each Fourier layer, and (c) is the v T structure.

[0022] Figure 3 The calculation steps of equation (7) are illustrated schematically.

[0023] Figure 4 The D-FNO solution used to solve the Poisson equation and the Darcy flow equation is shown. The first column shows an input test data, the second column presents the model output, the third column shows the true value, and the fourth column shows the pointwise absolute error between the model output and the true value.

[0024] Figure 5 Given a) the Poisson equation and b) the Darcy flow equation, along the two straight lines x =

[0025] Plot the D-FNO solution and the true solution u(x,y) with x = 0.5 and x = 0.8.

[0026] Figure 6The performance comparison of various neural network models in solving the Poisson equation (a) and the Darcy flow equation (b) is shown. To more clearly illustrate the changing trend of the training loss, the values ​​were smoothed by averaging a local window of 15 points.

[0027] Figure 7 The actual and predicted vorticity distributions at different time steps are shown, where the viscosity coefficient is v = 1e-4.

[0028] Figure 8 This is a line graph showing the vorticity values ​​predicted by FNO and D-FNO versus the actual values ​​at the 20th time step, x = 0.5 and y = 0.5, for three different viscosity coefficients.

[0029] Figure 9 This is a line graph showing the predicted vorticity values ​​and the actual values ​​of FNO and D-FNO at x=0.5 and y=0.5 for three different viscosity coefficients at a time step of 30.

[0030] Figure 10 This is a line graph showing the predicted vorticity values ​​and actual values ​​of FNO and D-FNO at x=0.5 and y=0.5 for three different viscosity coefficients at a time step of 40.

[0031] Figure 11 For three different viscosity coefficients, at a time step of 40, at x = 0.5 and y = 0.5, the improved D-FNO (labeled as D-FNO) is used. * A line graph showing the predicted vorticity values ​​versus the actual values.

[0032] Figure 12 The problem description of stress learning in a two-dimensional solid hyperelastic material is shown, where the input of the neural operator is the features of 972 non-uniform points, and the output is the stress at each point.

[0033] Figure 13 Predicted stress values ​​were obtained using Geo-FNO and Geo-DFNO in a two-dimensional hyperelastic example. The absolute error of the predicted stress value at each point is also plotted in the figure.

[0034] Figure 14 This is a schematic diagram of a three-dimensional elastic wave propagation problem, where the input of the neural operator is a velocity model, and the output is a time-varying surface velocity wave field obtained by upsampling from a virtual sensor grid.

[0035] Figure 15 The V values ​​predicted by FFNO and D-FNO at times 1.50 seconds, 1.74 seconds, and 2.74 seconds are... z The snapshot also shows the pointwise absolute errors of the two models.

[0036] Figure 16 The surface velocity V predicted by the FFNO and D-FNO models at two different locations is shown. z .

[0037] Figure 17 The D-FNO prediction results for the Poisson and Darcy flow equations using an alternative architecture are shown. The first column shows the input test data, the second column presents the model output, the third column shows the true values, and the fourth column shows the pointwise absolute error between the model output and the true values.

[0038] Figure 18 The table shows the training time and number of parameters for D-FNO and FNO in the Poisson equation, where the unit is seconds and the number in parentheses is the total number of parameters.

[0039] Figure 19 The table shows the inference times of D-FNO and FNO in the Poisson equation, where the unit is milliseconds, T1 is the total inference time, and T2 is the time spent in each Fourier layer during the inference process. Detailed Implementation

[0040] To enable those skilled in the art to better understand the technical solutions of this disclosure, exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments of this disclosure to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0041] Where there is no conflict, the various embodiments of this disclosure and the features thereof in the embodiments may be combined with each other.

[0042] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0043] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Words such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0044] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0045] Partial differential equations (PDEs) are fundamental to understanding various phenomena in physics, engineering, and other disciplines. Recently, advances in machine learning techniques, such as Fourier neural operators (FNOs), have shown promise for rapidly solving parameterized PDEs. By establishing a data-driven supervised learning framework, the mapping from the parameter space to the solution space in parameterized PDEs can be achieved. However, the Fast Fourier Transform (FFT) has considerable computational complexity, with a time complexity of O(N). 3 For large-scale 3D problems, training FNO is still very time-consuming (logN) (where N is the grid size in a single dimension).

[0046] Analysis of the complexity of FNO shows that the time complexity is mainly determined by the Fourier transform calculation, with a time complexity of O(MNQlog(MNQ)), where M, N, and Q are the sizes of the high-dimensional representations in the three spatial directions, respectively. As the problem size and complexity increase, the computational cost of FNO increases with the size and the number of Fourier layers, making it inefficient for solving large-scale three-dimensional problems.

[0047] This application aims to reduce the computational complexity of FNO by leveraging the separability of tensor decomposition and Fast Fourier Transform (FFT). Specifically, efficiency is improved by decomposing the hidden tensor into a series of one-dimensional tensor products and then performing an FFT on each one-dimensional tensor. This reduces the complexity of each FFT from O(MNQlog(MNQ)) to O(LlogL), where L = max(M, N, Q). Therefore, the proposed Decompositional Fourier Neural Operator (D-FNO) significantly improves the efficiency of existing FNOs in training and inference, particularly for large-scale three-dimensional problems.

[0048] In short, the contributions of this application are mainly reflected in three aspects:

[0049] (1) A decomposition Fourier neural operator is proposed to solve parameterized partial differential equations, that is, the high-dimensional hidden representation is decomposed into a set of one-dimensional tensor products.

[0050] (2) Through theoretical and numerical analysis, it was proved that the time complexity and the number of trainable parameters are reduced in the Fourier domain, which makes D-FNO more suitable for handling larger-scale problems.

[0051] (3) It demonstrates that D-FNO outperforms existing Fourier neural operator (FNO) methods in both steady-state and dynamic learning problems, and is effective for both two-dimensional (2D) and three-dimensional (3D) problems.

[0052] D-FNO aims to replace traditional solvers commonly used in Computer-Aided Engineering (CAE) software. D-FNO utilizes a data-driven approach to prediction, making it applicable to scenarios requiring extensive CAE software analysis. For example, it can be used for the design of engineering structures such as aircraft engine mounts and aircraft wings. By using D-FNO, the design process can be significantly accelerated, improving the efficiency of critical engineering assessments. D-FNO can also be applied to predicting global wind fields on the Earth's surface, a highly nonlinear dynamic problem influenced by solar radiation and topographic features. Predicting near-term surface winds is a complex challenge due to the dynamic nature of the influences involved. However, over 50 years of meteorological data is available, allowing D-FNO to be trained to effectively predict wind patterns. This capability is crucial for predicting potential typhoon tracks, enabling early disaster warnings and preparations to mitigate impacts.

[0053] 1. Fourier neural operator

[0054] The Fourier neural operator aims to learn the mapping from the input parameter function a(x) to the corresponding solution function u(x). The key idea of ​​the Fourier neural operator is to add a nonlocal integration operation during the regular network update process of each hidden layer, as shown in Equation (1).

[0055]

[0056] Where v t This is a high-dimensional representation of the solution at the t-th hidden layer, where W and σ represent the linear transformation and nonlinear activation function, respectively, and a represents the problem parameters. It is a nonlocal integral operator, defined as φ represents the network parameters. The convolutional nature of this integral allows it to be computed using the Fast Fourier Transform (FFT):

[0057]

[0058] in Represents the Fourier transform of the function. It is its inverse transform. R φ It is a tensor containing the kernel function κ. φ The Fourier coefficients are determined through network update learning.

[0059] The architecture of FNO is as follows Figure 1As shown. A local linear transformation W is added to each Fourier layer. Iterative updates. The definition is as follows:

[0060] First, the input a(x) is elevated to a high-dimensional representation v0 = P(a(x)) through a linear transformation P. Then, this high-dimensional representation is updated according to a series of equations (1) and (2): Finally, it is projected back to the physical solution through a linear transformation Q: u(x) = Q(v) T (x)).

[0061] 2. Decomposition Fourier Neural Operator

[0062] After analyzing the computational complexity of Fourier neural operators and Fast Fourier operators, it can be found that the computation of Fast Fourier Transform (FFT) dominates. Taking a two-dimensional problem as an example, the time complexity of FFT is O(MNlog(MN)), where M and N are the high-dimensional representations v. t Discrete grid sizes in the x1 and x2 spatial directions.

[0063] Inspired by tensor decomposition, this application proposes to transform the two-dimensional function v t It can be expressed as the sum of the products of two one-dimensional functions, i.e. It is not difficult to prove that at this time v t The Fourier transform of will be replaced by the sum of products of one-dimensional Fourier transforms, that is... In this way, the two-dimensional FFT is simplified to a one-dimensional FFT, and the time complexity becomes O(PMlog(M)) + O(PNlog(N)).

[0064] Tensor decomposition is a well-known technique in order reduction modeling. In most cases, the number of decomposition terms P is much smaller than M or N. Therefore, this method can significantly reduce computation time. This decomposition strategy can also be extended to three-dimensional problems, resulting in even greater computation time reductions.

[0065] To achieve separation in the inverse Fourier transform as well, the learnable kernel function is set to R. φ =R1·R2. At this point, the regional integral ∫ D 2 φ (xy,a)v t (y)dy becomes:

[0066]

[0067] Therefore, the computational complexity of the inverse Fourier transform is also O(PMlog(M)) + O(PNlog(N)).

[0068] 2.1 Architecture of Decompositional Fourier Neural Operator

[0069] The proposed two-dimensional D-FNO architecture is as follows: Figure 2 As shown. It follows a similar iterative process to FNO, but v t R φ The form and Fourier transform process differ. First, the input a is boosted to a high-dimensional representation v0 with a hidden channel size of H. Then, two different neural networks NN are used. r and NN c Decompose v0 into two one-dimensional vectors v 0,p,1 and v 0,p,2 (p = 1, 2, ..., P), where each vector has the same channel size. For example... Figure 2 As shown in (b), each of the two one-dimensional vectors undergoes multiple update iterations. They are then multiplied to generate P rank-1 matrices, and these matrices are then summed to obtain v. T ( Figure 2 c). Finally, the Q-neural network is used to transfer v T Project onto the desired two-dimensional output u.

[0070] Iterative update of each one-dimensional vector Defined as:

[0071]

[0072] Where t = 1, 2, ..., T are Fourier layer indices, d = 1, ..., D are dimension indices, and W... t,d It is a linear transformation of the d-th dimension. The final representation is v. T Calculated as

[0073] It should be noted that the decomposition method described above is not the only available method. Alternatively, the parameter function 'a' can be directly converted into a set of one-dimensional vectors as follows.

[0074] Consider the input as This is a general two-dimensional problem. Here, s1×s2 represents the input mesh size, and c represents the input feature size. First, we can obtain the reshaped... Then, v is calculated in the first direction as follows. 0,p,1 :

[0075]

[0076] Where e = 1, ..., s1, p = 1, ..., P, j = 1, ..., H, It is a linear transformation weight.

[0077] Similarly, in the second direction, the reshaped... v is calculated as follows: 0,p,2 :

[0078]

[0079] Where e = 1, ..., s2, p = 1, ..., P, j = 1, ..., H, These are the linear transformation weights. In the general case, after the linear transformation, v1 can be represented as a set of functions v1 = {v...} 1,p,d |d∈D,p∈P}, where D is the problem dimension. The other steps are calculated in the same way as D-FNO.

[0080] Under the same experimental setup, the total number of parameters for this new version of D-FNO is 2,233,025. The solutions for the Poisson equation and Darcy flow are as follows: Figure 17 As shown, the final test losses were 8.56e-4 and 2.047e-3, respectively. It can be seen that the results are comparable to those of the original D-FNO.

[0081] 2.2 Discrete Case and Fast Fourier Transform

[0082] Take a two-dimensional (D=2) problem with an s1×s2 grid as an example. The first layer will input... Upgraded to The size of the hidden dimension is H.

[0083] Then, the reduction layer is applied using the following formula. Transform into a set of row vectors with the same hidden dimension. and column vectors

[0084]

[0085] Where p = 1, ..., P and j = 1, ..., H. and They represent N and N respectively. r and NN c The weight.

[0086] In general, let the discretized grid be s1×s2×…×s D One-dimensional vector (where s) d The Fourier transform of (where the grid size is in the d-th dimension) is denoted as Similar to the Fourier neural operator, high-frequency modes are truncated to obtain Combine it with learnable kernel Multiplying them together, we get:

[0087]

[0088] Where k = 1, ..., k max And l = 1, ..., H. k max It is the sum of the preserved Fourier modes.

[0089] In the Fourier neural operator, the Fourier weight matrix R requires O(TH) time complexity. 2 k max D The proposed decomposed Fourier neural operator requires only O(TH) parameters in R. 2 k max D) parameters, which greatly reduces the number of parameters. The detailed calculation process of equation (7) is as follows: Figure 3 As shown.

[0090] Figure 3 This is a schematic diagram of the calculation steps of equation (7). For ease of explanation, a two-dimensional mesh with a size of 64×64 and a hidden dimension of 32 is used as an example. After the reduction layer, the shape of the latent representation becomes 64×1 with a hidden dimension of 32. The block on the left depicts the representation before multiplication. The block on the right shows the result of multiplication. exist Figure 3 In this model, all 64 patterns are preserved, and all representations of different items p share the same R.

[0091] For the d-th dimension as described in equation (4) The linear transformation gives the formula as follows:

[0092]

[0093] Where e = 1, ..., s d p = 1, ..., P and j = 1, ..., H are indices.

[0094] For hidden representation One-dimensional fast Fourier transform and inverse fast Fourier transform are defined as follows:

[0095]

[0096] Where p = 1, ..., P and j = 1, ..., H.

[0097] 2.3 Time Complexity Analysis

[0098] Taking an M×N two-dimensional mesh as an example, we will calculate the time complexity of FNO, FFNO, and the D-FNO of this application. For clarity, the calculation steps of each method are listed in Table 1.

[0099]

[0100] Table 1. Calculation steps for FNO, FFNO, and D-FNO. "NA" indicates not applicable. In FFNO, and These represent the directional Fourier transforms in the x and y directions, respectively.

[0101] The time complexity of each computational step is calculated and detailed in Table 2, where H is the hidden channel size, T is the number of Fourier layers, and P is the number of separation terms in the reduced layer. All Fourier modes were used in the calculation. Typically, P, H, and T are much smaller than the mesh size. Clearly, although the number of parameters in FFNO is reduced, its complexity is similar to that of FNO. Assuming M = N, for a two-dimensional problem, the time complexity of the most computationally intensive step (specifically, domain integration) is reduced from O(N) in FNO. 2 The time complexity is reduced from log(N)) to O(Nlog(N)) in D-FNO. For three-dimensional problems, this reduction is even more significant, with the time complexity decreasing from O(N)... 3 The efficiency of the proposed method is reduced from log(N) to O(Nlog(N)). This theoretical analysis demonstrates the efficiency advantage of the proposed method.

[0102]

[0103]

[0104] Table 2. Comparison of time complexity of FNO, FFNO, and D-FNO methods

[0105] 3. Numerical Experiment

[0106] A series of numerical experiments were conducted to verify the performance of D-FNO. The parametric equations handled included the two-dimensional steady-state Poisson equation and Darcy flow equation, the two-dimensional unsteady Navier-Stokes equation, the two-dimensional hyperelastic solid equation in irregular domains, and the three-dimensional elastic wave equation. These problems were selected from existing literature for comparative analysis with existing methods. All numerical simulations were performed on a single NVIDIA 4090 GPU with 24GB of memory and running Ubuntu 11.4.0. The optimization process used the Adam optimizer. The average relative L2 norm error between the predicted and target solutions was used as the loss function, defined as... in and ...

[0107] In all neural operators, the default number of Fourier layers is 4, using the ReLU activation function. The hidden dimension is typically 64, except for the Navier-Stokes equations, where the hidden dimension is 20 for FNO and 48 for D-FNO. Except for FFNO, which uses 24 Fourier modes, all other cases use a total of 12 Fourier modes. Unless otherwise explicitly stated, these parameters remain constant in all experiments. In all numerical experiments, a grid is used as an additional input.

[0108] 3.1 Two-dimensional steady-state problem

[0109] The Poisson equation is a classic partial differential equation widely used in electromagnetism and heat transfer. In this example, consider the two-dimensional Poisson equation over a unit square region.

[0110]

[0111] Dirichlet boundary conditions of u(x,y)=0 are applied to the boundaries at y=0 and y=1. Periodic boundary conditions are applied to the boundaries at x=0 and x=1. This example aims to learn the mapping f→u between the source and the solution. To generate the dataset, various source values ​​are sampled from a Gaussian random field with a covariance decay α of 2 and a scale parameter τ of 3. The corresponding numerical solutions are computed using the finite difference method (FDM) as the baseline truth values.

[0112] The two-dimensional Poisson equation can be applied to steady-state thermal conduction problems. For example, in the heat dissipation process of a mobile phone chip, the heat source intensity per unit volume corresponds to the parameter space of the two-dimensional Poisson equation, and the temperature field distribution corresponds to the solution space of the two-dimensional Poisson equation.

[0113] The Darcy flow equation is another fundamental model encountered in many applications. Considering the steady state of a two-dimensional Darcy flow, its form is:

[0114]

[0115] The domain is a unit cube, with Dirichlet boundary conditions u(x,y) = 0 applied on all boundaries. a(x,y) is the diffusion coefficient. In this problem, the mapping a → u between the diffusion coefficient and the solution is learned. The finite difference method (FDM) is used to compute the true solution u corresponding to various values ​​of a sampled from a Gaussian random field with a covariance decay α of 2 and a scale parameter τ of 3.

[0116] Darcy's flow equation can be applied to steady-state seepage problems of groundwater in porous media. In this example, the parameter space consists of permeability coefficients describing the ease or difficulty of fluid flow, and the solution space consists of hydraulic potential.

[0117] A total of 2000 samples were generated for the two two-dimensional steady-state problems mentioned above, of which 1800 samples were used for training and 200 samples were used for testing. The learning rate was set to 5e-3, the batch size was 50, the grid size was 64×64, and the number of decomposition terms P=25.

[0118] For the Poisson and Darcy flow examples, the mean and standard deviation of the relative L2 norm error over 200 test data points are 9.46e-4 ± 3.12e-4 and 2.24e-3 ± 5.62e-4, respectively. The error for each data point is calculated as follows: in and Let represent the predicted value and the true value of the i-th test data, respectively, both of which are expanded into one-dimensional vectors.

[0119] To visually represent the error distribution, Figure 4 The D-FNO solution for a test sample and its corresponding true value are plotted. The absolute error of the D-FNO solution is also shown. Clearly, the absolute error is approximately three orders of magnitude smaller than the true value. Furthermore, Figure 5 Line plots of the solutions at two different x-positions are shown, demonstrating excellent agreement between the D-FNO solutions and the true values. All these results collectively highlight the accuracy of D-FNO in both examples.

[0120] In the experiments described above, the grid was incorporated as an additional input, with a default P-value of 25. To investigate the impact of grid input and P-value on the accuracy of the method, D-FNO was retrained for different P-values ​​without using grid input. The average test loss for each case is listed in Table 3. Generally, increasing the P-value helps improve the model's learning ability, as evidenced by the decreasing trend of test loss at higher P-values. This is expected, as more decomposition terms provide more latent features for learning, resulting in more accurate results. However, this improvement plateaus when P reaches a certain value. Further increasing P does not significantly improve accuracy. In both cases, P=25 seems sufficient to achieve high accuracy. Subsequently, this value was set as the default in all numerical experiments. The results show that the accuracy level is comparable to other neural operators in all examples. Regarding the effect of grid input, it was observed that incorporating the grid helps reduce error.

[0121]

[0122] Table 3. Parameter study on the influence of grid input and P-value on the accuracy of D-FNO.

[0123] Next, the performance of D-FNO will be compared and evaluated with existing models (including FNO, FFNO, Resnet18, and U-Net). The FNO and FFNO versions used in this study are from the following code bases: https: / / github.com / zongyi- li / fourier_neural_operator and https: / / github.com / alasdairtran / fourierflow The learning rate for training these models was set to 1e-3. Performance evaluation was conducted at various grid sizes, specifically 32×32, 64×64, 128×128, and 256×256.

[0124] Figure 6 The first column shows the training loss curves of various neural networks trained on a 64×64 grid. In the Poisson equation and Darcy flow examples, FNO, FFNO, and D-FNO all exhibit similar convergence trends, and their loss decays significantly faster than U-Net and Resnet18, indicating that the Fourier neural operators have high effectiveness. Figure 6 The middle column plots the test errors of all models trained on different grids. The accuracy of the three Fourier neural operators remains consistent across different grids, with D-FNO showing the highest accuracy; in contrast, U-Net and ResNet18 have significantly lower prediction accuracy. Furthermore, the prediction performance of U-Net and ResNet18 further decreases as the grid size increases.

[0125] To evaluate the efficiency of all models, Figure 6 The last column plots the relationship between training time and the number of grid points for each model. Notably, with large grid sizes, D-FNO requires the shortest computation time among the three Fourier neural operators due to its lower time complexity. This efficiency advantage becomes more pronounced as the grid size increases, demonstrating its effectiveness in solving large-scale problems. While U-Net's training time is shorter than D-FNO's, its accuracy is significantly lower; to achieve the same level of accuracy, U-Net would require a much longer training time than D-FNO.

[0126] 3.2 Two-Dimensional Dynamic Problems

[0127] Consider the two-dimensional Navier-Stokes equations for a viscous incompressible fluid expressed in terms of vorticity (or vorticity) on a unit torus:

[0128]

[0129] Where u is the velocity field. Let w0 be the initial vorticity, and v be the viscosity coefficient. f(x) is the force term, fixed as f(x) = 0.1(sin(2π(x1+x2))+cos(2π(x1+x2))). The initial vorticity w0 follows a Gaussian random field with a covariance decay coefficient α of 2.5 and a scale parameter τ of 7. In this problem, the parameter space and solution space represent the vorticity at different times.

[0130] The operator that maps the vorticity of the first 10 time steps to the vorticity at a subsequent time T>10 is defined as... This is achieved by training a neural operator that receives the vorticity data from the first 10 time steps and predicts the vorticity at the next time step. Specifically, the vorticity at the 11th time step is predicted first using the initial 10 time steps of vorticity data, and then the vorticity at the 12th time step is predicted using the vorticity data from the 2nd to the 11th time steps. This iterative process continues, with the neural operator recursively applied, using the data from the first 10 time steps as input, to predict the vorticity up to time T.

[0131] A dataset was generated by solving the Navier-Stokes equations in stream function form using a pseudospectral method. This dataset contains 900 training samples and 100 test samples, corresponding to different initial vorticity distributions. The simulation time step is Δt = 1e-4, and the solution is recorded once every t = 1 time unit.

[0132] During training, the learning rate was set to 5e-3, and the batch size was 20. Training used data from time steps 11 to 20. Specifically, the loss value was calculated and the neural operator was trained using the actual and predicted vorticity data from time steps 11 to 20. The loss function was the average relative L2 norm error, defined as... in and Let be the predicted vorticity value and the actual vorticity value of the i-th sample at time step j, respectively. Let N represent the total number of samples, and ||*| represent the L2 norm. After training, this neural operator is recursively applied to predict the vorticity at time step 11 and thereafter until the final time point T.

[0133] First, consider the case where the viscosity is 1e-4. Figure 7The diagram shows the vorticity distribution of a randomly selected test sample at different time steps. The top row represents the true values, and the bottom row represents the prediction results of D-FNO. Visually, the two sets of results are highly consistent. The average point-by-point absolute error for different time steps is calculated as follows: 1.733e-3, 1.703e-3, 2.140e-3, and 7.738e-3 for time steps 15, 20, 30, and 40, respectively. Overall, the accuracy level at all time steps is satisfactory. Furthermore, it is worth noting that the error of the vorticity values ​​obtained through recursive iteration shows a slight upward trend with increasing time steps. This is due to the accumulation of errors in the iterative prediction process of D-FNO.

[0134] Subsequently, two additional scenarios with viscosity coefficients of 1e-3 and 1e-5 were explored, and the prediction results of D-FNO and FNO were compared. FNO has a hidden dimension of 20, while D-FNO has a hidden dimension of 48, indicating comparable parameter numbers. As before, the training loss was calculated using real and predicted vorticity data from time steps 11 to 20. On the three datasets, the test loss of FNO (expressed as mean relative L2 norm error) ranged from 0.1677 to 1.953; conversely, the test loss of D-FNO ranged from 0.1777 to 1.600. These test loss values ​​demonstrate that D-FNO performs comparably to FNO.

[0135] Figures 8 to 10 Line plots are shown at time steps 20, 30, and 40, at x = 0.5 and y = 0.5. Overall, in all three scenarios, the performance of D-FNO is very close to that of FNO, with its predictions matching the actual values ​​well, especially in the early time steps. However, at the 40th time step, or when the flow exhibits stronger chaotic characteristics, the predictions of D-FNO deviate significantly from the actual values, while FNO maintains its accuracy. This difference may be due to the insufficient number of decomposition terms used by D-FNO, preventing it from fully capturing complex flow patterns.

[0136] To address this issue, the total number of decomposition terms, P, was increased from the default 25 to 50, and the model was retrained every 75 rounds with a weight decay factor of 0.5. The results are as follows: Figure 11 As shown, the agreement between the two sets of results has been significantly improved, especially at the 40th time step.

[0137] To provide a more quantitative assessment of the accuracy of the two models, calculations were performed. Figures 8 to 11The average point-by-point absolute error of the prediction results shown is presented in Table 4. The improved D-FNO shows a reduction in error compared to the original D-FNO, which is particularly noticeable at the 40th time step. Further reduction in error can be achieved by further increasing the parameter P.

[0138]

[0139] Table 4. Average point-by-point absolute errors of FNO and D-FNO at different time steps. D-FNO * This indicates an improved D-FNO with a larger P-value.

[0140] 3.3 Two-dimensional problems with irregular shapes

[0141] In previous examples, all problem domains were constrained to regular domains to facilitate the creation of uniform meshes for Fast Fourier Transform (FFT) calculations. In this example, the performance of D-FNO on irregular physical domains is investigated. Specifically, a benchmark example of learning internal stress in two-dimensional hyperelastic materials is examined. The governing equations describing solid deformation are given by:

[0142]

[0143] Where ρ s Here, ρ is mass density, u is displacement, and σ is the stress tensor. The geometry of the domain is a unit square with an irregularly shaped cavity at the center, such as... Figure 12 As shown. The base of the unit square is fixed, while a tensile force of t = [0, 100] is applied to the top edge. The left and right sides are free. The radius of the cavity is determined by... The probability is determined, where The value ranges from 0.2 to 0.4. The solid material is an incompressible Rivlin-Saunders material with an energy density function parameter C1 = 1.863e 3 And C2 = 9.79e 3 The goal is to develop a neural operator that maps the shape of a void to stress values ​​at a series of non-uniform points within a solid. In this problem, the parameter space is the shape of the solid, and the solution space is the von Mises stress (equivalent stress).

[0144] The actual results were obtained using the finite element method. A dataset of 1000 data points was used for training, with an additional 200 data points reserved for testing.

[0145] By employing the Geo-FNO (Geometric-Perception-Natural Noise-Free) method, a deformable neural network φ is used. -1The irregular domain is transformed into a uniform grid. Then, a function applied to this uniform grid using the D-FNO method is taken as input. After completing the iterative Fourier process, an inverse deformable neural network φ is applied to restore the latent solution on the uniform grid to the original physical domain.

[0146] This paper denotes the implementation of D-FNO in general geometry as Geo-DFNO. The number of Fourier modes used is 12, with Geo-FNO and Geo-DFNO having hidden dimensions of 32 and 64 respectively. This results in a total of 1,482,657 parameters for Geo-FNO and 1,030,947 parameters for Geo-DFNO. Figure 13 The diagram plots the predicted stress and actual values ​​for multiple test samples, and also displays the absolute errors of the two neural operators at each point. It can be observed that Geo-DFNO achieves prediction results comparable to Geo-FNO.

[0147] 3.4 Three-dimensional elastic wave problem

[0148] To verify the effectiveness of D-FNO in solving three-dimensional equations, elastic wave propagation in a three-dimensional non-uniform medium was chosen as the test case. In this problem, the parameter space is the sound velocity in the medium, and the solution space is the wave field received by the sensor. This problem can be solved by... https: / / github.com / lehmannfa / HEMEW3D The obtained FFNO was simulated, so its performance can be directly compared with that of FFNO.

[0149] like Figure 14 As shown, this problem involves the propagation of elastic waves in a layered medium containing 2 to 7 horizontal layers. Except for the bottom layer, the thickness and shear wave velocity of each layer are randomly selected according to a pre-defined distribution, while the bottom layer remains consistent across all samples. A point source located at the bottom layer generates a wave whose radiation pattern is similar to that of the Lucien le Tälsch earthquake. This wave propagates through the medium, producing a unique surface wave field determined by the medium's composition.

[0150] The goal of this research is to develop a neural operator capable of predicting time-varying velocities at a 32×32 virtual sensor grid on the surface of the propagation domain, such as... Figure 14 As shown. The input to the neural operator is a velocity model discretely composed of 32×32×32 cubic pixels, representing a cubic rock with a side length of 9.6 km. Each voxel in the velocity model contains the shear wave velocity. The output is the time-varying surface wave velocity on a 32×32 uniform grid, with an output dimension of 32×32×320, where 320 represents the time step. The considered time interval is [1.0 s, 7.0 s], with a time increment of 0.02 s.

[0151] The dataset used in this example is from the HEMEW-3D database, containing 30,000 velocity models. The surface wavefield is obtained by solving the elastic wave equation. The D-FNO has a hidden dimension of 16 and contains 4 Fourier layers. The relative mean absolute error (MAE) is used as the loss function, and the training stopping condition is set to 250 epochs. Due to limited computational resources, only the vertical wave V in the z-direction is predicted. z Therefore, only a subset containing 3,600 training data points and 400 test data points was used, instead of the complete dataset.

[0152] Of the 400 test samples, the mean absolute error was 0.1262 for FFNO and 0.1183 for D-FNO. Figure 15 The two models demonstrate the surface velocity wave field V at three different time points for a single test sample. z The prediction results are as follows. Clearly, the D-FNO predictions closely match the actual values, while the FFNO predictions show a significant bias. Furthermore, Figure 16 The time histories of surface velocities predicted by the two models for two specific locations were plotted. Although both predictions deviate from the actual values, the D-FNO model significantly outperformed the FFNO model.

[0153] 3.5 Efficiency Study

[0154] This application investigates the efficiency of D-FNO. The analysis includes the number of parameters, training time, and inference time of D-FNO in solving the Poisson equation, Navier-Stokes equation, and three-dimensional elastic wave equation, and compares these results with those of FNO or FFNO. Furthermore, the impact of variations in the number of Fourier layers and grid size on model efficiency is examined.

[0155] Poisson equation

[0156] Fourier neural operators with different numbers of Fourier layers were trained and evaluated on Poisson equations with various grid sizes. Figure 18 Table 5 summarizes the average training time and standard deviation for the entire training dataset under each case, with the total number of parameters for each case given in parentheses below the training time. Inference time is listed in... Figure 19 In Table 6, T1 is the total inference time, and T2 is the time spent on computation in each Fourier layer, namely the time for Fast Fourier Transform (FFT), multiplication with matrix R, and Inverse Fast Fourier Transform (IFFT).

[0157] In both models, training time increases almost linearly with the number of layers, primarily due to the computational dominance of the Fourier layers. Notably, with smaller grid sizes and fewer layers, D-FNO's training time is longer than FNO's, likely due to the additional computational overhead of D-FNO. While training time increases with grid size, FNO's growth rate is higher than D-FNO's. For example, when the grid size increases from 64×64 to 512×512, FNO's average training time increases by 80.5 times, while D-FNO's only increases by 26.1 times. This difference means that at large grid sizes such as 256×256 and 512×512, D-FNO requires less total training time than FNO, demonstrating D-FNO's effectiveness in handling large-scale problems.

[0158] In this problem, the D-FNO has fewer parameters than the FNO, mainly due to the reduction in the number of parameters in the Fourier space (D-FNO has 49,152 parameters per Fourier layer, while FNO has 294,912 parameters per Fourier layer).

[0159] like Figure 19 As shown in Table 6, due to the reduced computation graph construction time, D-FNO typically spends less time on the Fourier layer (T2) than FNO. Similar to the trend in training time, as the grid size and the number of Fourier layers increase, the efficiency advantage of D-FNO becomes more pronounced, and its total inference time T1 is much shorter than that of FNO.

[0160] Navier-Stokes equations

[0161] To investigate the model's efficiency in the Navier-Stokes (NS) equations, data with a viscosity of 1e-4 and grid sizes of 64×64 and 128×128 were used as examples. Due to computational resource constraints, the experiments were not extended to larger grids. Table 7 shows the training time and number of parameters under different layer configurations. Notably, the training time of D-FNO exceeds that of FNO. Nevertheless, the relative difference between the two decreases with increasing number of layers and grid size. For example, in the case of a 64×64 grid and 2 Fourier layers, the relative difference in training time is close to 100%; while in the case of a 128×128 grid and 6 Fourier layers, this difference drops to about 35%. It can be expected that, similar to the observations in the Poisson equation, the training time of D-FNO may be less than that of FNO in scenarios with larger grid sizes and / or more Fourier layers.

[0162]

[0163] Table 7. Training time and number of parameters for D-FNO and FNO in the Navier-Stokes equations. Unit: seconds. The number in parentheses is the total number of parameters.

[0164] Table 8 shows the inference times for solutions with a prediction time step of 20 under different conditions. Clearly, on a 128×128 grid, the total inference time of D-FNO is lower than that of FNO. With further increases in grid size, the inference time advantage of D-FNO is expected to become even more pronounced.

[0165]

[0166]

[0167] Table 8. Inference times for D-FNO and FNO in the NS equations. Unit: milliseconds. T1 is the total inference time. T2 is the time spent in each Fourier layer during inference.

[0168] Three-dimensional elastic wave equation

[0169] Since the training data for this example comes from a public database with a grid size limited to 32×32×32, the focus is primarily on the impact of the number of Fourier layers. In this example, the 32×32×32 three-dimensional velocity model is mapped to a 32×32×320 time-varying surface velocity profile, where 320 is the time step. Therefore, the third dimension of the tensor expands after each Fourier layer, with dimensions of [64,320], [64,128,320], [64,128,256,320], [64,128,256,256,320], and [64,128,128,256,256,320] for 2 to 6 Fourier layers, respectively. The number of Fourier modes used along the third dimension is 16 in the initial Fourier layer and 32 in all subsequent layers.

[0170] Table 9 records the training time and the number of parameters. Although D-FNO has more parameters than FFNO, its training time is significantly shorter, and the gap widens further with the number of layers. With 6 Fourier layers, the training time of D-FNO is 1 / 7.5 that of FNO. This efficiency mainly stems from the significant reduction in time complexity per Fourier layer, from O(N) to O(FFNO). 3 The efficiency advantage of D-FNO is reduced from logN to O(NlogN). The efficiency advantage of D-FNO is again reflected in the inference time, as shown in Table 10. For D-FNO, the time spent on Fourier layer computation (T2) accounts for only a small portion of the total inference time (T1), while in FFNO, T2 accounts for a large portion of T1.

[0171]

[0172] Table 9. Training time and number of parameters for D-FNO and FNO in the three-dimensional elastic wave equation. Unit: seconds. The number in parentheses is the total number of parameters.

[0173]

[0174] Table 10. Inference times for D-FNO and FNO in the three-dimensional elastic wave equation. Unit: milliseconds. T1 is the total inference time, and T2 is the time spent in each Fourier layer during the inference process.

[0175] In the three-dimensional example, the efficiency advantage of D-FNO is more significant compared to the two-dimensional case, which is consistent with the theoretical analysis of time complexity. For the two-dimensional problem, the time complexity of Fourier layer computation decreases from O(N^2) to O(N^2). 2 The value of log(N) is reduced to O(Nlog(N)). This property is particularly advantageous for solving large-scale three-dimensional parametric equations, such as optimization problems that require repeated calls to the solver to solve the underlying partial differential equations.

[0176] This application proposes a Decomposed Fourier Neural Operator (D-FNO), which aims to reduce the training and inference time of FNO and FFNO by leveraging the separability of tensor decomposition and Fast Fourier Transform (FFT). For a three-dimensional problem of size M×N×Q, the time complexity of the Fourier layer computation is reduced from O(MNQlog(MNQ)) to O(LlogL), where L = max(M,N,Q). Furthermore, by incorporating a parameter sharing mechanism, the number of parameters in the Fourier layer is also reduced compared to FNO.

[0177] Furthermore, various numerical experiments were conducted across different domains to evaluate the effectiveness of D-FNO. The results show that D-FNO's accuracy is comparable to FNO or FFNO, but its efficiency is significantly improved, especially in large-scale complex problems. In the most challenging three-dimensional elastic wave propagation experiment, compared to FFNO, D-FNO reduced training time by up to 7.6 times and inference time by 7.0 times, while maintaining slightly better prediction accuracy. For problems with mesh sizes larger than 32×32×32 and / or more than 6 Fourier layers, the efficiency advantage of D-FNO is expected to be even more significant. In addition, its ability to handle irregular point cloud inputs was verified through a two-dimensional hyperelastic example.

[0178] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. A computer-implemented method of solving a parametric partial differential equation, comprising: lifting a parametric function to a high-dimensional representation; decomposing the high-dimensional representation into a plurality of one-dimensional vectors with the same hidden dimension; iteratively updating each of the plurality of one-dimensional vectors with a plurality of Fourier layers cascaded in sequence; multiplying the iteratively updated plurality of one-dimensional vectors to obtain a plurality of rank-1 matrices, and summing the plurality of matrices to obtain a final representation; and projecting the final representation back to a low-dimensional physical interpretation. The decomposing the high-dimensional representation into a plurality of one-dimensional vectors with the same hidden dimension comprises decomposing the high-dimensional representation into the plurality of one-dimensional vectors with a plurality of different neural networks.

2. The method of claim 1, wherein, Iterative update of each one-dimensional vector is defined as: where t = 1, 2,... T is the Fourier layer index, d = 1,..., D is the dimension index, p = 1,..., P is the decomposition term index, W t,d is the linear transformation of the dth dimension, σ denotes the non-linear activation function, denotes the Fourier transform of the function, is its inverse transform, R t,d is the learning kernel.

3. The method of claim 1, wherein, The partial differential equation comprises a two-dimensional steady-state Poisson equation and Darcy flow equation, a two-dimensional unsteady Navier-Stokes equation, a two-dimensional hyperelastic solid equation in irregular domain, and a three-dimensional elastic wave equation.

4. The method of claim 1, wherein, The parametric function is discretized to grid data before being lifted to the high-dimensional representation.

5. The method of claim 1, wherein, 6. An electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores one or more computer programs executable by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 5. The computer program, when executed by the processor, implements the method of any one of claims 1 to 5.

7. A computer readable storage medium having stored thereon a computer program, wherein, ​

Images