Physical attention-enhanced Fourier neural operator for three-dimensional turbulence prediction

Abstract

The invention belongs to the technical field of three-dimensional turbulence prediction, and particularly relates to a physical attention-enhanced Fourier neural operator for three-dimensional turbulence prediction. The PAFNO uses the idea of FNO to capture global information and uses a physical attention mechanism for three-dimensional turbulence intrinsic physical correlation complementation. The PAFNO captures the global information of the three-dimensional turbulence through fast Fourier transform, learns the internal physical information of the three-dimensional turbulence through a physical attention mechanism module, carries out physical information complementation, and carries out the global information complementation through residual connection and secondary fast Fourier transform. Compared with a two-dimensional turbulence problem, the three-dimensional turbulence problem is more complex, the calculation requirement is harsh, the PAFNO accurately captures the physical characteristics contained in the three-dimensional turbulence, and the method is superior in seven data sets.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional turbulence prediction, and particularly relates to a physical attention enhanced Fourier neural operator for three-dimensional turbulence prediction. Background Art

[0002] In the field of computational fluid dynamics, it has also become a focus to use models such as deep neural networks (DNNs), neural operators (NOs), and Transformers to replace traditional methods for solving problems. Deep neural networks have shown impressive performance in approximating high-dimensional non-linear functions. For example, the physics-informed neural network (PINN) achieves the goal of learning physical information by introducing equation information into the loss function and has currently been applied in fields such as fluid dynamics, singular perturbation differential equations, and seepage mechanics. After the advent of neural operators, the mapping between finite-dimensional Euclidean spaces was topologized to the mapping between infinite-dimensional function spaces. The most widely known Fourier neural operator (FNO) effectively improves the computational efficiency by introducing the fast Fourier transform (FFT), opening up new ideas. Subsequently, we have seen models such as FFNO and Geo-FNO that attempt to solve the N-S problem, and U-FNO that attempts to solve the multiphase flow problem. As a powerful and popular model, the Transformer has recently also been used in the solution of partial differential equations, but directly applying it to the calculation of a huge number of grid points will face the difficulty of low computational efficiency. Based on the idea of the Transformer, the Transolver and the LNO (Latent Neural Operator) have respectively proposed a physical attention mechanism and a physical cross-attention mechanism, making changes to the core attention mechanism module in the Transformer and achieving surprising results.

[0003] In the prior art, most of the problems in the field of computational fluid dynamics using deep learning models stay at one-dimensional and two-dimensional problems. However, due to the reality of physical problems, fluid problems in nature, especially turbulence problems, are mostly three-dimensional problems. Three-dimensional turbulence can better reflect its complex non-linear characteristics and can also better reveal the universal characteristics of turbulence.

[0004] Three-dimensional turbulence has become one of the most complex and challenging problems in fluid mechanics due to its strong non-linearity, uncertainty, etc. Moreover, a huge amount of computing power resources will be consumed during the calculation of three-dimensional turbulence. In recent years, with the development of deep learning and the progress of GPU computing power, using data-driven artificial intelligence methods to calculate three-dimensional turbulence has become a powerful tool. Summary of the Invention

[0005] The present invention proposes a physical attention enhanced Fourier neural operator for three-dimensional turbulence prediction, which solves the problems of low solution accuracy and high computational cost in current three-dimensional turbulence prediction.

[0006] The technical solution of the present invention is achieved in this way: Physical attention-enhanced Fourier neural operator for 3D turbulence prediction, including the following: S1. Up-dimensionalize the input data, map the velocity field information in the three-dimensional turbulence into a high-dimensional feature space, and capture the global features of the three-dimensional turbulence from the initial time to the final time and the local features in each time step; S2, using Fourier neural operators to capture global features through Fourier transform and inverse transform; S3, physical attention mechanism enhancement; S4, state change correction; S5, residual link; Repeat steps S2 to S5, and finally output the predicted three-dimensional turbulent velocity field.

[0007] Through the above technical solution, step S1 upgrades the data to a high-dimensional space, not only discretizes the data, but also allows the model to directly approximate the value of the data, therefore, not only reduces the computational cost, accelerates the model calculation, but also facilitates the subsequent prediction operation of the model. Step S2 can ensure the discrete invariance in the high-dimensional space through the Fourier transform and its inverse transform in the Fourier neural operator, so that the data can be directly learned in the high-dimensional space, and the high-speed performance of the Fourier transform itself can not only improve the accuracy of model learning, but also reduce the computational cost. And due to the existence of discrete invariance, the model will not be like the traditional model, and the error will be superimposed in large quantities during the training process, which specifically solves the problem of low solution accuracy in three-dimensional turbulence prediction. In step S3, the physical attention mechanism has the characteristics of being able to learn the intrinsic physical state hidden behind the discreteness, and can learn the data details ignored by the Fourier neural operator, thereby improving the calculation accuracy of the model. The state change correction in step S4 is similar to the classic numerical integration methods such as the explicit Euler method, which makes the model state update process more physically meaningful and explanatory. In complex systems such as three-dimensional turbulence, it can combine the time scale and the law of state change to effectively describe the physical characteristics, thereby improving accuracy. Step S5 introduces residual links, adds inputs and outputs during training, supplements the data details lost during training, alleviates the gradient vanishing and gradient exploding problems during training, and ensures the accuracy of the model.

[0008] Optionally, in step S1, data After input, the dimension must be upgraded first, and the input velocity field information , through local transformation Lifted to a higher-dimensional representation, which is parameterized by a shallow fully-connected neural network or an equivalent convolutional layer: ; Among them, is a sequence of value-taking functions, is the three-dimensional turbulent initial velocity field data input to the model, , is a local transformation for dimension elevation, acting independently on each spatial component of the function , is the input velocity field information, is a separable Banach function space, is the corresponding real number field set in is the input sequence the corresponding real number field set in is the corresponding value taken, that is, represents the corresponding input data , is a local transformation acting independently on each spatial component of the function, and the output is the projection of through the local transformation, is the output of the corresponding function sequence, is a local transformation for dimension reduction, and outputs a sequence of functions.

[0009] Optionally, in step S2, let the input data pass through a Fourier neural operator for feature recognition; ; Among them, and are the Fourier transform and the inverse Fourier transform, is a linear complement, indicating that the input data has passed through the Fourier neural operator.

[0010] Through the above technical solution, the data passes through the Fourier neural operator, is lifted to the high-dimensional Fourier space, and captures the global characteristics of the three-dimensional turbulence changing from the initial time to the end time through the Fourier transform and the inverse Fourier transform, ensuring that the model can learn the overall change law of the three-dimensional turbulence over time and improving the accuracy of the model.

[0011] Optionally, in step S3, make the input data After passing through the physical attention mechanism module, the output ; Assume a grid set , containing grid points, which are embedded into deep features through a linear layer, and each grid point contains channels, and contains geometric information and physical information, , define a paradigm, according to the features learned by each grid point divide each grid point into potential slices, formalized as follows: ; ; Among them, Project() is a pointwise linear layer adapted to general geometries, projecting channels into weights, and obtaining slice weights after Softmax(), and Softmax() is a normalized exponential function, represents the slice weight, represents the i-th deep feature, represents the j-th slice in the i-th grid point, represents the j-th slice feature; the above formula divides each grid point into M potential slices, and grid points with the same physical state are grouped into the same slice, and by calculating the attention to the slice encoding, the physical associations under complex geometric conditions are effectively captured; Then each slice is encoded into a physically aware token through spatial weighted aggregation, which can be written in the following form: ; Among them, , represents the i-th token feature; represents the j-th token feature; represents the j-th slice feature in the i-th grid point; represents the j-th slice in the i-th grid point; represents the i-th deep feature; the above formula encodes slices with physically internal consistency through spatial weighted aggregation, so that each token contains specific physical state information; An attention mechanism is used between the re-encoded tokens to capture the complex associations between different physical states, that is ; Among them, , represents the query vector, represents the key vector, represents the value vector, represents the token feature after the softmax operation, is a linear transformation, represents the token feature to be linearly transformed, represents the matrix transpose; the above formula realizes the feature recognition function of the network model through the operations of linear transformation and softmax; Recombine the slice weights and label, transform back to the grid points, which can be written as: ; where , represents the j-th token feature; the above formula enables each physical perception token to be broadcast to all grid points in the above calculation process; The whole process can be expressed as: ; represents the data after the feature recognition of the physical attention mechanism, and the complete is expressed as: ; where, represents the physical attention mechanism, represents the data the data output after the feature recognition of PhyAtten.

[0012] Through the above technical solution, the purpose of step S3 is to make the input data after passing through the physical attention mechanism module, output , and make up for the features of three-dimensional turbulence through the physical attention mechanism, so that the model architecture can more accurately perform the feature recognition of three-dimensional turbulence.

[0013] Optionally, in step S4, ; where, is the time step, is a correction function.

[0014] Through the above technical solution, the data has passed through to correct the state change, and obtain the rate and law of the model state change.

[0015] Optionally, in step S5, a skip connection is introduced to directly add the input and the output, alleviating the problems of vanishing gradients and exploding gradients during training. Assume that an input in a deep learning model is , which undergoes a non-linear transformation and the output is . Then the output can be expressed as: ; wherein, the input after passing through is added to the original input for input feature complementation; ; wherein, corresponds to in the above formula, corresponds to in the above formula, corresponds to in the above formula. This formula represents that the data undergoes feature recognition by a physically attention-enhanced Fourier neural operator, then undergoes for state change correction, and finally the original input data is added in the form of a residual connection to obtain the output result .

[0016] Through the above technical solution, step S4 is similar to classical numerical integration methods such as the explicit Euler method, making the process of model state update more physically meaningful and interpretable. In addition, due to the non-linear characteristics of σ, theoretically, introducing will be very compatible with non-linear problems and multi-scale problems. In complex systems such as three-dimensional turbulence, it can combine the time scale and the state change law to effectively describe physical characteristics.

[0017] Optionally, the predicted three-dimensional turbulence velocity field is output as: ; wherein, represents that the data undergoes feature recognition by a Fourier neural operator, represents that the data undergoes feature recognition by a physically attention-enhanced Fourier neural operator.

[0018] After adopting the above technical solution, the beneficial effects of the present invention are: The present invention proposes a Fourier neural operator PAFNO integrated with a physical attention mechanism, aiming to solve the three-dimensional turbulence problem in the case of high Reynolds numbers (referring to a very strong turbulent flow state, the higher the Reynolds number, the stronger the turbulence). Because the Reynolds number is too high, it makes the three-dimensional turbulence difficult to predict. PAFNO captures the global information of three-dimensional turbulence through the fast Fourier transform, then learns the intrinsic physical information of three-dimensional turbulence through the physical attention mechanism module to supplement the physical information, and then makes up for the global information through residual connection and the second fast Fourier transform.

[0019] A novel Fourier neural operator PAFNO of the present invention can still accurately capture the complex physical information contained in three-dimensional turbulence under the condition of few data samples, providing a new idea for realizing three-dimensional turbulence simulation by using low-cost methods. The present invention integrates the physical attention mechanism into the Fourier integral operator to supplement the internal physical correlation while ensuring the rapid capture of the global information of three-dimensional turbulence, perfectly balancing the division of labor between the FFT and the physical attention mechanism.

[0020] The PAFNO model in the present invention first raises the data to a high-dimensional space, not only discretizes the data, but also enables the model to directly approximate the value of the data. Therefore, it not only reduces the calculation cost and accelerates the model calculation, but also facilitates the subsequent prediction operation of the model; in the high-dimensional space, through the Fourier transform and its inverse transform in the Fourier neural operator, the discrete invariance can be guaranteed, enabling the data to be directly learned in the high-dimensional space. Coupled with the high-speed performance of the Fourier transform itself, it not only improves the accuracy of model learning, but also reduces the calculation cost. And due to the existence of discrete invariance, the model will not, like traditional models, have a large amount of error superposition during the training process, specifically solving the problem of low solution accuracy in three-dimensional turbulence prediction; using the physical attention mechanism, it can learn the characteristics of the internal physical state hidden behind the discretization, and can learn the data details ignored by the Fourier neural operator, improving the calculation accuracy of the model; using state change correction makes the process of model state update more physically meaningful and interpretable. In a complex system such as three-dimensional turbulence, it can combine the time scale and the state change law to effectively describe the physical characteristics, thereby improving the accuracy; by introducing residual links, adding the input and output during the training process to supplement the data details lost during the training process, alleviating the problems of gradient disappearance and gradient explosion during training, and ensuring the accuracy of the model. Brief Description of the Drawings

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0022] Figure 1 It is a schematic diagram of the PAFNO architecture in the embodiment; Figure 2 It is a schematic diagram of the 2D slicing result of dataset 1 in the embodiment; Figure 3 It is a schematic diagram of the 2D slicing result of dataset 2 in the embodiment; Figure 4 It is a schematic diagram of the 2D slicing result of dataset 3 in the embodiment; Figure 5 It is a schematic diagram of the 2D slicing result of dataset 4 in the embodiment; Figure 6 It is a schematic diagram of the 2D slicing result of dataset 5 in the embodiment; Figure 7 It is a schematic diagram of the 2D slicing result of dataset 6 in the embodiment; Figure 8 It is a schematic diagram of the 2D slicing result of dataset 7 in the embodiment; Figure 9 It is the architecture of PAFNO-A and PAFNO-B in the embodiment. Detailed implementation manners

[0023] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0024] The embodiments of the present application disclose a physically attention-enhanced Fourier neural operator for three-dimensional turbulence prediction.

[0025] Embodiment, according to Figures 1 to 9 As shown, the physically attention-enhanced Fourier neural operator for three-dimensional turbulence prediction includes the following.

[0026] 1. Related work 1.1 Fourier transform and Fourier neural operator The Fourier transform and its inverse transform are the basis of the Fourier neural operator and are also a powerful mathematical tool for the frequency decomposition and reconstruction of signals. It converts signals between the time domain and the frequency domain. The Fourier transform has wide applications in signal processing, fluid mechanics, image analysis, and machine learning. Its wide application makes frequency-domain analysis one of the important methods for solving complex problems.

[0027] (1) (2) Among them, represents the Fourier transform, represents the inverse Fourier transform, is the natural logarithm, is pi, is the imaginary unit, represents the value for which the Fourier transform is performed, represents the value for which the inverse Fourier transform is performed, is the differential with respect to , is the differential with respect to .

[0028] The FNO (Fourier neural operator) captures the global features of the input through the Fourier transform. This global perspective is particularly important for simulating physical systems with long-range dependencies such as turbulence. In addition, in the frequency domain, the interaction of global information can be achieved through simple point multiplication; (3) Equation (3) is the mathematical expression of the frequency-domain convolution theorem, the equivalence relationship between time-domain convolution and frequency-domain multiplication, that is, convolution is equivalent to point multiplication in the frequency domain, where and are two different signals, represents the Fourier transform corresponding to the signal, represents and the Fourier transform after multiplying the two signals, represents the signal performing the Fourier transform, represents the signal performing the Fourier transform, represents multiplication, represents point multiplication, and global feature extraction can be completed at the complexity of , reducing the computational complexity while achieving the fusion of global features.

[0029] Thus, the FNO reconstructs a novel spectral method, directly defining a new convolution operator in the Fourier space to replace the original local convolution operation: (4) (5) Among them, is the function to be transformed, represents the Fourier transform, represents the inverse Fourier transform, represents the th operation, is the domain of the function , is the natural logarithm, is the circumference ratio, is the imaginary unit, represents the value for performing the Fourier transform, represents the value for performing the inverse Fourier transform, is the differential with respect to , is the differential with respect to .

[0030] After directly parameterizing in the Fourier space, the Fourier integral operator can be defined as: (6) Among them, represents the result after applying frequency-domain convolution and then transforming back to the physical space, is the information of the input three-dimensional turbulent velocity field, represents the corresponding information in the th layer, represents the Fourier transform, represents the inverse Fourier transform, is the learning parameter corresponding to , is the domain of the transform, represents the position of the information of the input three-dimensional turbulent velocity field in the domain . is a mathematical symbol, represents that it holds for all , is a mathematical symbol representing belonging to. represents performing the Fourier transform on the feature represented by .

[0031] Finally, the iterative formula of FNO is written as: (7) Among them, represents the feature of the information of the input three-dimensional turbulent velocity field in the th layer, Represents the position of the input three-dimensional turbulent velocity field information in the domain, Represents the correction function, Is a linear transformation, Represents the input three-dimensional turbulent velocity field information at the Features in the layer, Represents the result after applying frequency-domain convolution and then transforming back to the physical space, Represents the input three-dimensional turbulent velocity field information, Represents the boundary condition, Represents defined as, Represents the Features of the position information in the layer Represents the features of the position information represented by Represents the Features of the position information in the layer Represents the features of the position information represented by

[0032] 1.2 State change correction In IFNO (Implicit Fourier Neural Operator), Is used to correct the change of the model state. Among them, Is the time step, which represents the discretization of the model in time and is used to control the advancement amplitude of the model in each iteration. Can be regarded as a non-linear mapping or a correction function, such as ReLu or GeLu, which expresses the rate and law of the model state change and depends on the internal weights of the model or the current state of the model. When they are combined, Can express the correction of the model state change, where Determines the amplitude of the model evolution, Represents the evolution trend of the current state of the model and is used to update the state F(x) of the model. Each time the model is updated and iterated, The influence of will be accumulated into F(x) to predict the state of the next time step. Thus, after using The formula of the FNO model can be written as: (8) Among them, Represents the Layer, Represents the Layer, Is the time step, Is the correction function, Represents the position of the input three-dimensional turbulent velocity field information in the domain, Represents the features corresponding to the three-dimensional turbulent velocity field information, Is a linear transformation, Represents the result after applying frequency-domain convolution and then transforming back to the physical space. Represents the input three-dimensional turbulent velocity field information. Represents the boundary conditions Represents the model architecture. Represents the model architecture of the Fourier neural operator. Represents defined as Represents the features of the position information represented in the th layer Represents the features of the position information represented in the th layer Represents the model architecture of the th layer of the Fourier neural operator.

[0033] This mechanism is similar to classical numerical integration methods such as the explicit Euler method, making the process of model state update more physically meaningful and interpretable. In addition, due to the non-linear characteristics, theoretically, introducing will be very compatible with non-linear problems and multi-scale problems. In complex systems such as three-dimensional turbulence, it can combine time scales and state change laws to effectively describe physical characteristics.

[0034] 1.3 Attention mechanism The attention mechanism is a technique widely used in deep learning to improve the model's ability to capture key information. It was initially proposed for natural language processing tasks and later extended to fields such as computer vision, time series prediction, speech processing, and has recently shown great promise in solving partial differential equations. The attention mechanism enables the model to focus on important task-related information and ignore irrelevant parts by assigning weights to different parts of the input data. The attention function can be described as mapping a query and a set of key-value pairs to an output, where the query, key, value, and output are all vectors. The output is calculated as a weighted sum of the values, where the weight assigned to each value is calculated by a compatibility function between the query and the corresponding key: (9) where represents the attention mechanism, represents the query vector, represents the key vector, represents the value vector, represents the normalization exponential function, represents the dimension of the key vector, represents the matrix transpose.

[0035] 2. Methods 2.1 Overall architecture Both the standard FNO and the attention - based deep learning models have achieved good results so far, but they are not all - powerful. For example, FNO usually truncates high - frequency information and only retains low - frequency features. When dealing with the detailed structure of three - dimensional turbulence or the fine changes in the boundary layer, it may lose key information and reduce the prediction accuracy. Moreover, the computational complexity of the FFT (Fourier transform) itself is , which is efficient. However, in 3D problems, the memory and computational requirements of the FFT will increase significantly; attention - based deep learning models almost all have a common fatal flaw - high computational cost and dynamic memory cost. During model training and weight storage, it will bring great troubles to scenarios with limited hardware resources. Moreover, the attention mechanism lacks the ability of inductive bias and will struggle when dealing with small - sample data sets. Therefore, the present invention proposes a new deep learning model PAFNO, specifically as Figure 1 shown.

[0036] The initial idea of the PAFNO model is similar to that of FNO. It elevates the input data to a higher - dimensional space, maps it to a high - dimensional space, captures global features through FFT and inverse transform, then uses a physical attention module to supplement the physical features of three - dimensional turbulence, and then captures the three - dimensional turbulence features again through a Fourier transform module, enabling the model to fully learn the internal physical correlations of the problem. In addition, PAFNO also uses the method of residual connection to promote gradient flow and improve the ability of the model to reuse features. Moreover, the method of implicit recursion can intuitively reduce the repeated definition of parameters and the number of loops, enabling the model to run more efficiently on the GPU. Generally speaking, the PAFNO model can be written in the following form: (10) (11) Where, represents the th layer, represents the th layer, is the time step, is the correction function, represents the features corresponding to the three - dimensional turbulence velocity field information, represents the information of the th layer after feature recognition by the physically - attention - enhanced Fourier neural operator, represents the physical attention mechanism, is a linear transformation, and represent the Fourier transform and the inverse Fourier transform respectively, represents the model architecture, represents the model architecture of the physically - attention - enhanced Fourier neural operator, Represents the linear transformation of the layer, is the velocity field feature information of the layer, represents the velocity field feature information of the

[0037] 2.2 Dimensionality Ascension and Projection The purpose of dimensionality ascension is to map the input low-dimensional physical variables to a high-dimensional feature space. In the problem scenario of the present invention, it can be described as mapping the velocity field information in three-dimensional turbulence to a high-dimensional feature space so as to capture the complex global and local features of three-dimensional turbulence. The input velocity field information is first lifted to a higher-dimensional representation through a point-by-point (local) transformation This operation is usually performed by a shallow fully connected neural network or an equivalent convolutional layer with parameters: (12) Among them, is a sequence of value-taking functions, is the initial velocity field data of three-dimensional turbulence input to the model, , is the local transformation for dimensionality ascension, acting independently on each spatial component of the function , is the input velocity field information, is a separable Banach function space, is the corresponding real number field set in is the input sequence the corresponding real number field set in is the corresponding value-taking, that is, represents the corresponding input data , is acting independently on each spatial component of the function , and the output is the projection of through the local transformation, is the output of the corresponding function sequence, the local transformation for dimensionality reduction, the sequence of output functions.

[0038] 2.3 Physical Attention First, introduce the concept of learning physical perception tokens. Assume a grid set contains grid points, and embed them into deep features through a linear layer and each grid point contains channels, and contains geometric and physical information, written as . Define a normal form, according to each grid point learned features Each grid point is dissected into potential slices, formalized as follows: (13) (14) where Project() is a pointwise linear layer adapted to general geometries, projecting channels into weights, and obtaining slice weights after Softmax() , Softmax() is the normalized exponential function, represents the slice weight, represents the i-th depth feature, represents the j-th slice in the i-th grid point, represents the j-th slice feature; Then each slice is encoded into a physically aware token through spatial weighted aggregation, which can be written in the following form: (15) where, represents the j-th token feature; represents the j-th slice feature in the i-th grid point; represents the j-th slice in the i-th grid point; represents the i-th depth feature.

[0039] An attention mechanism is adopted among the re-encoded tokens to capture the complex associations between different physical states, that is (16) where, , represents the query vector, represents the key vector, represents the value vector, represents the token feature after the normalized exponential function operation, is a linear transformation, represents the token feature to be linearly transformed, represents the matrix transpose, represents the value vector; Subsequently, the slice weights are recombined into tokens, and is transformed back to the grid point, which can be written as: (17) Among them , represents the j-th marked feature.

[0040] The whole process can be summarized as: (18) represents data after feature recognition by the physical attention mechanism, and the complete is expressed as: (19) Among them, and are the Fourier transform and the inverse Fourier transform, is linear complementation, represents the input data, represents the physical attention mechanism.

[0041] 2.4 Other components The PAFNO model also uses the technique of residual connection. The residual connection was first proposed by He et al. By introducing a skip connection, the input is directly added to the output, thus alleviating the problems of gradient vanishing and gradient explosion in training. Assume that an input in a deep learning model is , and its output after passing through a non-linear transformation is , then the output can be expressed as (20) In the present invention, there is an embodiment of the residual connection in and in formulas (10) and (11).

[0042] Among them, in formula (10), the one corresponding to is , the one corresponding to is , and the one corresponding to is ; in formula (11), the one corresponding to is , the one corresponding to is , and the one corresponding to is .

[0043] 3. Experiments and Results 3.1 Problem Setting The dimensionless Navier-Stokes equations in conservation form for three-dimensional incompressible turbulence are as follows: (21) (22) where is the velocity, is the value of the modified pressure divided by the constant density, is the Reynolds number, is the large-scale force that maintains the statistical stationarity of the turbulence. Regarding the initial velocity field , the present invention uses 7 different initial conditions to generate a data set. The data generation method uses the pseudo-spectral method to numerically simulate an incompressible three-dimensional turbulence with a uniform grid size of 64*64*64 on a cube. The velocity field can be expanded into a Fourier series; (23) where represents the imaginary unit, ; represents the wave number vector, represents the velocity in Fourier space, and the hat represents the variable in wave number space. The incompressible Navier-Stokes equations in Fourier space are: (24) represents the wave number vector, represents the velocity in Fourier space.

[0044] (25) where and are wave number vectors, is the th component. The time step of each data set is set to , the total duration , and the Reynolds number is fixed at 100000.

[0045] 3.2 Training Settings To better demonstrate the performance of the model proposed by the present invention, the present invention compares PAFNO with a total of three models, namely FNO, IFNO, and Transolver. Among them, Transolver first proposed the physical attention mechanism and showed the best performance in several other tasks, demonstrating strong generalization ability. To ensure the fairness of the experiment, the present invention uses the same settings for PAFNO and the other three models, as shown in Table 1: Table 1 Training Settings

[0046] To ensure that each model is fully trained during the training process and does not waste too much computing time and resources, the number of training epochs is uniformly set to 500; to avoid excessive occupation of video memory resources and ensure training efficiency at the same time, the batch size of training is uniformly set to 10; the initial learning rate of each model participating in training is uniformly set to 0.003, and the Adam optimizer is uniformly used for parameter update and learning rate adjustment, with weight decay being 。

[0047] 3.3 Datasets and Results For a total of seven datasets, the present invention shows their different initial velocity field representations in Table 2.

[0048] Table 2 Velocity field settings for seven datasets

[0049] Among them, is the intensity of the vortex ring, set to 5.0, R is the radius of the vortex ring, set to 1.0, r is the radial distance, and the calculation method is: (26) The present invention shows its different large-scale forces F in Table 3.

[0050] Table 3 Large-scale force settings for seven datasets

[0051] Among them, represents the noise amplitude, set to 0.1, represents generating a uniform random number matrix with the same shape as , represents the gravitational acceleration, set to 9.81.

[0052] Four model error numerical results in seven datasets are given here first, as shown in Table 4 specifically: Table 4 L2 errors of numerical experiments for seven datasets

[0053] 3.3.1 Dataset 1 Dataset 1 is in the form of a typical Taylor - Green vortex, with a periodic and symmetric structure. Although the initial velocity component in the direction is , the present invention applies a force with the same periodicity in this direction. The experimental results are specifically as Figure 2As shown. The experimental result figures are presented by slicing the final frame of the last training epoch. From Figure 2 it can be seen that due to the complexity of Dataset 1 itself, FNO and IFNO can only learn approximate global information and perform poorly in depicting local information. Transolver and PAFNO achieve visibly better results compared to FNO and IFNO, but Transolver is inferior to PAFNO in depicting local information.

[0054] 3.3.2 Dataset 2 Different from Dataset 1, the large-scale force in Dataset 2 is changed to random perturbation forces in three directions. The specific experimental result figures are as Figure 3 shown. It can be seen that all four models perform well in Dataset 2, but the results of Transolver and PAFNO are better, and the result of PAFNO is also the best.

[0055] 3.3.3 Dataset 3 In Dataset 3, the present invention introduces a constant gravity into the z-direction component of the large-scale force. In terms of the numerical values of the Loss of the four models, Dataset 3 is not a very difficult dataset. The two models FNO and IFNO achieve acceptable results, but still cannot match Transolver and PAFNO. Moreover, PAFNO still obtains the best result on Dataset 3. The 2D slice results of Dataset 3 are as Figure 4 shown.

[0056] 3.3.4 Dataset 4 In Dataset 4, the initial velocity field information is defined as a vortex ring structure, and like Dataset 1, a periodic force is applied in the direction. The result in Dataset 4 is still that PAFNO is the best, Transolver is the second best, and FNO and IFNO perform poorly. It is speculated that the reason for the oblique distribution shown by FNO and IFNO in the error image is that they only perform Fourier transform and its inverse transform, without performing attention supplementation like Transolver and PAFNO. The specific results are as Figure 5 shown.

[0057] 3.3.5 Dataset 5 Dataset 5 is similar to Dataset 4, with the difference that the large-scale force in Dataset 5 becomes a constant gravity. The similarity between Dataset 5 and Dataset 4 can also be seen from the numerical results, and PAFNO still wins the first place, Transolver comes second, and IFNO and FNO rank third and fourth. The 2D slice results of Dataset 5 are as Figure 6 shown.

[0058] 3.3.6 Dataset 6 In Dataset 6, the initial velocity component in the z - direction is also set to a non - zero initial value, and a constant gravity is used to regulate the evolution process. In the numerical results, PAFNO also won in Dataset 6. In addition, observing the 2D slice plot as Figure 7 shown, PAFNO obviously learned the physical features contained in Dataset 6 better.

[0059] 3.3.7 Dataset 7 For Dataset 7, the initial velocity components in the three directions are completely randomly set without any pattern. Although it is difficult to find such an example in a real physical scenario, during the data generation process, the constraints brought by Equation (20) and Equation (21) are still followed, and the generated dataset still contains the physical information of three - dimensional turbulence. In a completely random experiment, if PAFNO can also win, it can better demonstrate the strong generalization ability of PAFNO, as shown specifically in Figure 8 the figure.

[0060] From the numerical results, although the dataset generated completely randomly is of low difficulty, PAFNO indeed demonstrated its strong generalization ability. It still achieved the best results on Dataset 7. The three models Transolver, FNO, and IFNO also achieved satisfactory results on Dataset 7.

[0061] 3.4 Ablation Experiments To verify the optimal architecture of the PAFNO model, the present invention designed two groups of ablation experiments. In the first group of ablation experiments, the Fourier integral operator block with a physical attention mechanism was looped, which is called PAFNO - A; in the second group of ablation experiments, the physical attention mechanism was placed inside the second Fourier integral operator block, which is called PAFNO - B. Figure 9 The specific architectures of the two models PAFNO - A and PAFNO - B are shown.

[0062] The present invention uses Dataset 1 for verification. The numerical results of the ablation experiments show that the PAFNO model has the optimal architecture. The specific results are shown in Table 5 as follows: Table 5 Results of Ablation Experiments

[0063] The present invention proposes a Fourier neural operator enhanced by a physical attention mechanism to handle small-sample three-dimensional turbulent datasets in high Reynolds number cases. The model uses the idea of FNO to capture global information and uses a physical attention mechanism to supplement the intrinsic physical correlations of three-dimensional turbulence. Numerical experiments show that the PAFNO model has achieved superiority on seven datasets and obtained a relative gain of 40.20%, providing a new direction for three-dimensional turbulent simulations with limited computing power. In the more complex three-dimensional turbulent problems with more demanding computational requirements compared to two-dimensional turbulent problems, PAFNO accurately captures the physical characteristics contained in three-dimensional turbulence and has achieved superiority on seven datasets.

[0064] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the technical solutions of the present invention shall be included within the protection scope of the present invention.

Claims

1. Physical attention enhanced Fourier neural operator for 3D turbulence prediction, characterized by: Includes the following: S1. Up-dimensionalize the input data, map the velocity field information in the three-dimensional turbulence into a high-dimensional feature space, and capture the global features of the three-dimensional turbulence from the initial time to the final time and the local features in each time step; S2, using Fourier neural operators to capture global features through Fourier transform and inverse transform; S3, physical attention mechanism enhancement; S4, state change correction; S5, residual link; Repeat steps S2 to S5, and finally output the predicted three-dimensional turbulent velocity field.

2. The physical attention enhanced Fourier neural operator for three-dimensional turbulence prediction according to claim 1, characterized in that: In step S1, data After input, the dimension must be upgraded first, and the input velocity field information , through local transformation To a higher dimensional representation, this operation is performed by a shallow fully connected neural network or equivalent The convolutional layer is parameterized: ； in, is a sequence of value functions, is the three-dimensional turbulent initial velocity field data input to the model, , It is a local transformation that increases the dimension and acts independently on the function Each spatial component of , is the input velocity field information, is a separable Banach function space, yes The corresponding real number field set in , is the input sequence The corresponding real number field set in , yes The corresponding value is Represents the corresponding input data , It is independent of the function For each spatial component of yes Projection via local transformation , is the output of the corresponding function sequence, Local transformations that reduce dimensionality, Outputs a sequence of functions.

3. The physical attention enhanced Fourier neural operator for three-dimensional turbulence prediction according to claim 2, characterized in that: In step S2, let the input data After Fourier neural operator, feature recognition is performed; ； in, and is the Fourier transform and inverse Fourier transform, is a linear complement, Represents input data The output result after feature recognition by Fourier neural operator.

4. The physical attention enhanced Fourier neural operator for three-dimensional turbulence prediction according to claim 3, characterized in that: In step S3, the input data After passing through the physical attention mechanism module, the output ; Assume a grid set ,Include grid points, embedding them into deep features through linear layers , and each grid point contains channels, and contains geometric and physical information. , define a paradigm, according to each grid point Learned features Each grid point Divide into potential slices, formalized as follows: ； ； Among them, Project() is a point-by-point linear layer that adapts to general geometric shapes. The channels are projected onto Among the weights, the slice weight is obtained after Softmax() , Softmax() is a normalized exponential function, represents the slice weight, represents the i-th deep feature, represents the jth slice in the i-th grid point, represents the jth slice feature; Each slice is then encoded into a physics-aware token via spatially weighted aggregation, which can be written as follows: ； in, , represents the i-th marker feature; represents the jth marker feature; represents the jth slice feature in the i-th grid point; represents the jth slice in the i-th grid point; represents the i-th deep feature; The attention mechanism is used between the re-encoded tokens to capture the complex associations between different physical states, that is, ； in, , represents the query vector, represents the key vector, represents a numeric vector, represents the label feature after the normalized exponential function operation, is a linear transformation, Indicates the marker features that need to be linearly transformed, Represents matrix transpose; Regroup the slice weights and mark them Transformed back to grid points, it can be written as: ； in, , represents the jth marker feature; The whole process can be expressed as: ； Representative data After the feature recognition of the physical attention mechanism, the complete It is expressed as: ； in, represents the physical attention mechanism, Representation data Data output after PhyAtten feature recognition.

5. The physical attention enhanced Fourier neural operator for three-dimensional turbulence prediction according to claim 4, characterized in that: In step S4, ； in, is the time step, is a correction function.

6. The physical attention enhanced Fourier neural operator for three-dimensional turbulence prediction according to claim 5, characterized in that: In step S5, a skip connection is introduced to directly add the input and output to alleviate the gradient vanishing and gradient exploding problems in training. Assume that one of the inputs in the deep learning model is , which undergoes a nonlinear transformation The output after , then output It can be expressed as: ； Among them, the input In passing After transformation, add the original input Complement input features; ； in, Corresponding to the above formula , Corresponding to the above formula , Corresponding to the above formula , this formula represents the data The Fourier neural operator enhanced by physical attention is used for feature recognition, and then Make state change corrections, and finally add the original input data using residual links , and the output result is .

7. The physical attention enhanced Fourier neural operator for three-dimensional turbulence prediction according to claim 6, characterized in that: Output predicted three-dimensional turbulent velocity field : ； in, Indicates that the data is identified by Fourier neural operator. Representation data is subjected to feature recognition by Fourier neural operators enhanced by physical attention.

Images