Flow field prediction method based on self-supervised learning and neural operator

Abstract

The invention particularly relates to a flow field prediction method based on self-supervised learning and a neural operator, which realizes efficient flow field feature extraction and prediction by adopting a'pre-training-fine tuning 'two-stage strategy and combining a mask image modeling method and a Fourier neural operator architecture in the self-supervised learning. In a pre-training stage, features of flow field data are extracted through a coding network, and generalization ability of a model is enhanced based on self-supervised learning of mask image modeling; in the fine tuning stage, a double-input FNO neural operator is designed and used to carry out rapid and high-precision prediction on flow field evolution. The method not only retains the advantage of easy training of the FNO model, but also improves the stability and applicability of the model by strengthening the feature extraction network, and significantly improves the accuracy and efficiency of flow field prediction.

Description

Technical Field

[0001] The present invention relates to the technical field of computational fluid dynamics, and in particular to a flow field prediction method based on self-supervised learning and neural operators, which is used to efficiently and accurately predict the flow field evolution process in fluid dynamics. Background Art

[0002] In recent years, with the rapid development of the field of artificial intelligence, data-driven deep learning methods have received widespread attention and application in computational fluid dynamics. For flow field prediction, deep learning methods use large-scale flow field data to train neural network models through proxy modeling, which can learn the approximate solution of the entire system and then predict the evolution of the flow field. Since such methods do not require complex spatiotemporal discretization of partial differential equations, after training, the speed of flow field simulation is several orders of magnitude higher than that of traditional methods. In addition, thanks to the powerful generalization ability of neural networks, such methods can, on the one hand, predict the flow field at any time during the evolution process, and on the other hand, for control conditions (boundary conditions, physical properties, geometric shapes, etc.) not included in the training data, predictions of a certain degree of accuracy can be made without retraining.

[0003] In some recent research, scholars have embedded physical mechanisms and theoretical modeling into neural networks, designing a series of methods with higher numerical simulation accuracy, such as the Physical Information Neural Network (PINN), Deep Operator Network (DeepONet), and Fourier Neural Operator (FNO). Although existing research has achieved significant results in flow field prediction, these methods still have some limitations. Related work can be roughly divided into two categories: approximate solution function methods and approximate solution operator methods.

[0004] The approximate solution function method is a deep learning framework for solving partial differential equations in physics. In 2019, Raissi et al. proposed the PINN model in the paper "RAISSIM, PERDIKARIS P, KARNIADAKIS G E. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 2019." This model uses deep learning to infer physical laws and optimize network parameters. Based on a feedforward neural network, this framework approximates the solution of partial differential equations by learning the distribution of training data, while simultaneously minimizing a loss function based on physical laws. For the approximate solution function method, the training process is the solution process. Therefore, changing the parameters of the equation or the initial conditions requires retraining, which is difficult to train and has poor generalization performance.

[0005] Unlike approximate solution function methods, approximate solution operator methods learn the entire solution family by learning the mapping from input functions to output functions. In 2019, Lu et al. proposed the Deep Operator Network (DeepONet) in the paper "LU L, JIN P, KARNIADAKIS G E. Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators[J]. arXiv preprint, 2019." In this paper, they explained the approximation operator, which has wider applicability and better explanatory power than the approximate solution function method. In 2021, Li et al. introduced the Fourier Neural Operator (FNO) in the paper "LI Z, KOVACHKI N, AZIZZADENESHELI K, et al. Fourier neural operator for parametric partial differential equations. arXiv preprint, 2020." Each layer of the Fourier Neural Operator is divided into two branches. The first branch uses fast Fourier transform technology to process the low-frequency features captured by the fully connected network, and then uses inverse fast Fourier transform technology to further process this low-frequency information. The second branch passes it directly through the fully connected layer and merges it with the output of the previous branch. The purpose of this is to maintain high-frequency information. Since only complex calculations are performed on low-frequency information, this method significantly improves the calculation speed, making it a highly efficient neural network. Compared with previous deep learning methods such as convolutional neural networks and physical information neural networks, approximate solution operator methods are more accurate and faster when predicting flow fields. However, methods such as DeepONet require shallow networks to approximate simulation operators, so the training cost is relatively high. Although the FNO method has improved in terms of training cost, its performance is not stable enough and its generalization performance needs to be improved.

[0006] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute prior art known to ordinary technicians in this field. Summary of the Invention

[0007] The present invention provides a flow field prediction method based on self-supervised learning and neural operators, which is used to solve the problems of high training cost, poor generalization performance and instability faced by existing deep learning methods in flow field prediction.

[0008] Other features and advantages of the present invention will become apparent from the following detailed description, or may be learned in part by practice of the present invention.

[0009] According to a first aspect of the present invention, a flow field prediction method based on self-supervised learning and neural operators is provided, the method comprising:

[0010] Obtaining original flow field velocity field data, performing normalization processing on the original flow field velocity field data; dividing the normalized data into blocks, and performing mask processing on the block data to obtain masked flow field data;

[0011] Constructing a self-supervised pre-training network, wherein the self-supervised pre-training network uses the Transformer model as the backbone network to construct an encoder-decoder autoencoding model;

[0012] Pre-train the self-supervised pre-training network: use the masked flow field data as input and the complete flow field data as supervision information to train the network parameters to learn the flow field characteristics;

[0013] Use the pre-trained encoder to extract features from the flow field data, map it to the same dimension as the flow field data through a fully connected layer, and freeze its parameters;

[0014] A dual-input FNO neural operator network is designed. It takes the current flow field data and extracted flow field features as input, and uses future flow field data as supervision information. The dual-input FNO neural operator network is trained based on the regression task, and its parameters are continuously optimized until convergence.

[0015] The flow field data at the current moment is input into the trained dual-input FNO neural operator network to obtain the flow field data at the next moment.

[0016] In some exemplary embodiments, performing mask processing on the block data specifically includes:

[0017] In each round of training, some blocks are randomly masked to simulate the scene of local information loss in the flow field and construct the masked flow field data.

[0018] In some exemplary embodiments, the encoder-decoder autoencoder model, wherein the encoder is a 12-layer Transformer, adopts a multi-head attention mechanism with 8 heads, and the input is a sequence of masked flow field blocks; the decoder adopts a lightweight convolutional network, which is responsible for reconstructing the flow field values of the masked area, and the output of the decoder is the restored flow field data.

[0019] In some exemplary embodiments, the autoencoder model uses mean square error and contrast loss to define the loss function for mask region reconstruction, wherein the mean square error is used to measure the difference between the reconstructed flow field data and the original flow field data, and the contrast loss is used to enhance the distinguishability of the flow field representation generated by the encoder.

[0020] In some exemplary embodiments, the flow field data at the current moment and the extracted flow field features are used as input, specifically:

[0021] The flow field data at the current moment is concatenated with the features extracted by the encoder according to the channel dimension. The generated fusion features are then input into the Fourier neural operator for frequency domain transformation and nonlinear mapping.

[0022] According to a second aspect of the present invention, a storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the flow field prediction method based on self-supervised learning and neural operators described in the first aspect is implemented.

[0023] According to a third aspect of the present invention, a computer program product is provided, on which a computer program is stored. When the computer program is executed by a processor, the flow field prediction method based on self-supervised learning and neural operators described in the first aspect is implemented.

[0024] According to a fourth aspect of the present invention, there is provided an electronic device, comprising:

[0025] processor; and

[0026] a memory for storing executable instructions of the processor;

[0027] Wherein, the processor is configured to implement the flow field prediction method based on self-supervised learning and neural operators described in the first aspect above by executing the executable instructions.

[0028] The flow field prediction method based on self-supervised learning and neural operators provided by the embodiments of the present invention significantly improves the accuracy and efficiency of flow field prediction by combining the flow field prediction method of self-supervised learning and neural operators. The flow field feature extraction model based on self-supervised learning proposed by the present invention enhances the characterization ability of the feature encoding network for flow field data through the mask image modeling method, so that the network can effectively learn the local and global features under different flow phenomena and control conditions, thereby improving the accuracy of flow field prediction. In addition, the dual-input Fourier neural operator architecture combines the feature encoding network and flow field data as dual-branch inputs, further supplementing the flow field evolution information, thereby improving the prediction ability of the model. Through a large number of experimental verifications, the method of the present invention shows better performance than other existing methods in the flow field prediction of multiple typical flow phenomena, especially when dealing with complex flow problems. It has significant advantages.

[0029] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings are incorporated into and constitute a part of this specification, illustrate embodiments consistent with the present invention, and together with the description, serve to explain the principles of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and it is clear that those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0031] Figure 1 A specific flow chart for the implementation of the present invention;

[0032] Figure 2 This is the network structure diagram of the present invention. DETAILED DESCRIPTION

[0033] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0034] In addition, the accompanying drawings are merely schematic illustrations of the present invention and are not necessarily drawn to scale. Identical reference numerals in the figures denote identical or similar parts, and thus repetitive descriptions thereof will be omitted. Some of the blocks shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0035] In response to the shortcomings and deficiencies of the existing technology, this example embodiment provides a flow field prediction method based on self-supervised learning and neural operators. This method adopts a two-stage strategy of "pre-training-fine-tuning" and combines the mask image modeling method and Fourier neural operator architecture in self-supervised learning to achieve efficient flow field feature extraction and prediction. In the pre-training stage, the features of the flow field data are extracted through the encoding network, and the generalization ability of the model is enhanced by self-supervised learning based on mask image modeling; in the fine-tuning stage, a dual-input FNO neural operator is designed and used to quickly and accurately predict the evolution of the flow field. This method not only retains the advantage of the FNO model being easy to train, but also improves the stability and applicability of the model by strengthening the feature extraction network, significantly improving the accuracy and efficiency of flow field prediction.

[0036] refer to Figure 1 As shown, the method of the present invention may specifically include the following steps:

[0037] Step 1: Flow field data preprocessing and self-supervised feature construction: The raw flow field velocity data is normalized and divided into blocks. In each round of training, some blocks are randomly masked to construct the masked flow field data.

[0038] Step 2: Pre-training the self-supervised feature extraction network: Using the Transformer model as the backbone network, we construct an encoder-decoder autoencoder model. During training, we use the masked flow field data as input and the complete flow field data as supervision information to train the network parameters and learn the flow field features.

[0039] Step 3: Flow field feature encoding: Use the pre-trained encoder network to extract features from the flow field data and freeze its parameters to ensure that the feature extraction network does not participate in parameter updates during subsequent training.

[0040] Step 4: Fourier Neural Operator (FNO) flow field prediction model training: Design a dual-input FNO neural operator architecture, take the current flow field data and extracted flow field features as input, and use the flow field data at future moments as supervision information. Train the FNO model based on the regression task and continuously optimize its parameters until convergence.

[0041] Step 5, flow field prediction: Input the flow field data at the current moment into the flow field prediction model trained in the previous step to quickly obtain the flow field data at the next moment.

[0042] Below, each step in this exemplary implementation will be described in more detail with reference to the accompanying drawings and embodiments.

[0043] Step 1: flow field data preprocessing and self-supervisory feature construction.

[0044] (1a) The original flow field velocity data is normalized and mapped to the [-1, 1] interval.

[0045] (1b) The flow field is divided into local blocks of 16×16 grids. 30% of the blocks are randomly masked in each round of training to simulate the scene of local information loss in the flow field and construct the mask input U masked .

[0046] Step 2: Self-supervised pre-training based on mask reconstruction.

[0047] (2a) Construct a self-supervised pre-training network based on Vision Transformer. The network adopts an encoder-decoder structure, the encoder E is a 12-layer Transformer, the hidden layer dimension = 768, and a multi-head attention mechanism with 8 heads. The input is the mask flow field block sequence U masked The decoder D uses a lightweight convolutional network to reconstruct the flow field value of the masked area. The output of the decoder is the restored flow field data

[0048]

[0049] (2b) The loss function for mask region reconstruction is defined using mean square error and contrast loss. The mean square error is used to measure the difference between the reconstructed flow field data and the original flow field data, especially the masked area:

[0050]

[0051] Where U true,i is the original flow field data, M i It is the element in the mask matrix, indicating which areas need to be reconstructed.

[0052] Contrastive loss is used to enhance the discrimination of the flow field representations generated by the encoder, ensuring that similar flow field representations are closer in the feature space and different flow field representations are farther away. Contrastive loss is calculated based on cosine similarity:

[0053]

[0054] Where z i and z j is the representation of the flow field generated by the encoder, sin(·,·) is the cosine similarity function, and τ is the temperature parameter. The final loss function is a weighted combination of mean square error loss and contrast loss:

[0055]

[0056] Where λ is the weight hyperparameter of the contrastive loss. By combining mean squared error and contrastive loss, the reconstruction effect and representation ability of the model are optimized.

[0057] (2c) Use the Adam optimizer with an initial learning rate of 1e-4 and iterate for 50,000 steps until the model converges and save the network model parameters.

[0058] Step 3: Flow field feature encoding.

[0059] (3a) Load all the parameters of the encoder part E of the converged model in the previous step, freeze its parameters, and only retain its feature extraction capability.

[0060] (3b) Taking the flow field data U(t) at time t as input, the features extracted by the encoder are used to map it to the same dimension as the flow field data through the fully connected layer.

[0061] F1=FC(E(U(t)))(5)

[0062] Step 4: fine-tune the dual-input Fourier neural operator.

[0063] (4a) Construct a dual-input Fourier neural operator. Branch 1 is the original flow field data U(t). Branch 2 is the feature F1 extracted by the encoder in the previous step. The outputs of the two branches are concatenated according to the channel dimension and passed through the convolution layer to generate the fused feature. This process can be expressed as:

[0064] F final =Conv(concat(U(t),F1)) (6)

[0065] (4b) The fused feature F final Input Fourier neural operator (FNO) for frequency domain transformation and nonlinear mapping:

[0066]

[0067] Where, The core operation of FNO is defined as:

[0068]

[0069] Where, is the Fourier transform, W k is the learnable frequency domain weight matrix, and b(x) is the spatial domain bias term.

[0070] (4c) Map the features to the output dimension through the neural network Q and calculate the loss:

[0071]

[0072] The network is trained with this loss until convergence, and the model parameters are saved.

[0073] Step 5: Flow field prediction.

[0074] (5a) Load all the parameters of the converged model in the previous step.

[0075] (5b) The flow field data at the current moment to be predicted is input into the trained dual-input Fourier neural operator network to quickly obtain the flow field data at the next moment.

[0076] The effects and advantages of the present invention can be further verified through the following experiments.

[0077] Experimental environment

[0078] The present invention is an experiment conducted using the pytorch framework on an Intel(R) Xeon(R) Silver 4214R CPU @ 2.40GHz, 16G memory, and a Linux operating system.

[0079] The dataset used in the experiment comes from the CFDBench dataset, proposed by Xia et al. in the paper "Y Luo, Y Chen, Z Zhang. CFDBench: A Comprehensive Benchmark for Machine Learning Methods in Fluid Dynamics. Preprints.org, 2023." CFDBench includes benchmarks for four classic problems in computational fluid dynamics: lid-driven square cavity flow, laminar boundary layer flow in a circular tube, dam flow over a step, and periodic Karman vortex street. Each flow problem contains data with different boundary conditions, fluid properties, and flow domain geometry. All data were generated using ANSYS Fluent 2021R1 software. Each data subset was divided into training, validation, and test sets in an 8:1:1 ratio. Splitting the data ensures that operating parameters in one group do not appear in other groups.

[0080] Experimental content

[0081] Extensive experiments were conducted on the public CFDBench dataset, testing the performance of the self-supervised learning-based flow prediction method and existing cutting-edge algorithms using velocity field data from four typical flow phenomena: cavity flow, pipe flow, dam flow, and cylindrical flow. Comparative experimental results demonstrate that the proposed algorithm outperforms other similar methods in terms of flow prediction (using cavity flow as an example, as shown in Table 1). Ablation experiments also demonstrate the performance improvement achieved by the self-supervised pre-training strategy (using dam flow and pipe flow as examples, as shown in Table 2).

[0082] Table 1 Flow field prediction performance under cavity flow

[0083]

[0084] Table 2 Ablation experiment: The impact of self-supervised pre-training on flow field prediction

[0085]

[0086] Other embodiments of the present invention will readily occur to those skilled in the art after considering the specification and practicing the invention herein. This application is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the invention being indicated by the claims.

[0087] It should be understood that the present invention is not limited to the exact construction described above and shown in the drawings and that various modifications and variations can be made without departing from the scope thereof, which is limited only by the appended claims.

Claims

1. A flow field prediction method based on self-supervised learning and neural operators, characterized in that: The method comprises: Obtaining original flow field velocity field data, performing normalization processing on the original flow field velocity field data; dividing the normalized data into blocks, and performing mask processing on the block data to obtain masked flow field data; Constructing a self-supervised pre-training network, wherein the self-supervised pre-training network uses the Transformer model as the backbone network to construct an encoder-decoder autoencoding model; Pre-train the self-supervised pre-training network: use the masked flow field data as input and the complete flow field data as supervision information to train the network parameters to learn the flow field characteristics; Use the pre-trained encoder to extract features from the flow field data, map it to the same dimension as the flow field data through a fully connected layer, and freeze its parameters; A dual-input FNO neural operator network is designed. It takes the current flow field data and extracted flow field features as input, and uses future flow field data as supervision information. The dual-input FNO neural operator network is trained based on the regression task, and its parameters are continuously optimized until convergence. The flow field data at the current moment is input into the trained dual-input FNO neural operator network to obtain the flow field data at the next moment.

2. The method according to claim 1, characterized in that The masking process for the block data is specifically as follows: In each round of training, some blocks are randomly masked to simulate the scene of local information loss in the flow field and construct the masked flow field data.

3. The method according to claim 1, characterized in that The encoder-decoder autoencoder model is a 12-layer Transformer that uses a multi-head attention mechanism with 8 heads. The input is a sequence of masked flow field blocks. The decoder uses a lightweight convolutional network to reconstruct the flow field values of the masked area. The output of the decoder is the restored flow field data.

4. The method according to claim 3, characterized in that The autoencoder model uses mean square error and contrast loss to define the loss function for mask region reconstruction, where the mean square error is used to measure the difference between the reconstructed flow field data and the original flow field data, and the contrast loss is used to enhance the distinguishability of the flow field representation generated by the encoder.

5. The method according to claim 1, wherein The flow field data at the current moment and the extracted flow field features are used as input, specifically: The flow field data at the current moment is concatenated with the features extracted by the encoder according to the channel dimension. The generated fusion features are then input into the Fourier neural operator for frequency domain transformation and nonlinear mapping.

6. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the flow field prediction method based on self-supervised learning and neural operators according to any one of claims 1 to 5 is implemented.

7. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the flow field prediction method based on self-supervised learning and neural operators according to any one of claims 1 to 5 is implemented.

8. An electronic device, characterized in that: include: processor; as well as a memory for storing executable instructions of the processor; The processor is configured to execute the flow field prediction method based on self-supervised learning and neural operators according to any one of claims 1 to 5 by executing the executable instructions.

Images