Structural adjustment type neural operator modeling method

Abstract

The invention discloses a neural operator modeling method with a structure adjustment capability and a multi-stage optimization mechanism. The neural operator modeling method is suitable for efficiently solving a complex partial differential equation under a small sample condition. According to the method, a primary function regulation and control network is provided, a traditional fixed primary function is replaced, and the function representation capacity is dynamically adjusted through a structure guiding module; in the aspect of feature fusion, a collaborative modulation mechanism is introduced, and deep semantic integration of input conditions and space coordinates is realized. The training process adopts a staged design, and comprises structure adaptive preheating, self-supervised reconstruction and physical consistency collaborative optimization, so that the precision, stability and interpretability of a solution are effectively improved. Experimental verification shows that the method has good generalization and physical rationality in the problems of Darcy flow, Poisson equations and the like, and is suitable for scientific calculation tasks of complex boundaries, multi-scale modeling and the like.

Description

Technical Field

[0001] The present invention relates to a method for solving partial differential equations based on deep neural networks, and specifically to a neural operator modeling method and its training framework that integrates an attention mechanism, learnable POD basis functions, and physical regularization constraints. This method belongs to the field of high-performance physical modeling technology at the intersection of scientific computing, artificial intelligence, and numerical simulation, and is particularly suitable for industrial physics problems that require high-precision prediction of spatial distribution solutions such as velocity, pressure fields, and temperature fields. Background Art

[0002] Partial differential equations (PDEs) are widely used in natural science and engineering computations. They are an important mathematical expression for describing continuous changes in space and time. Common PDEs include the Poisson equation, the heat conduction equation, the elastic wave equation, the Navier-Stokes equations, and Darcy's law. They play a central role in fields such as climate modeling, subsurface fluid flow, microelectronics design, and complex material simulation.

[0003] For a long time, the numerical solution of PDEs has relied primarily on mesh discretization methods such as the finite element method (FEM), the finite difference method (FDM), and the finite volume method (FVM). These methods rely on high-quality meshing to map the continuous solution domain into discrete nodes or elements. However, as the complexity of simulation problems increases, these methods have gradually exposed the following limitations:

[0004] Curse of dimensionality: In high-dimensional space, the number of discrete points increases exponentially with the dimension, greatly increasing computational overhead.

[0005] Strong mesh dependency: Complex geometries or deforming domains require frequent mesh reconstruction, which limits the scalability of the method.

[0006] Difficulty in reconstructing when boundaries change: When initial / boundary conditions change, traditional methods usually require reconstructing a new system from scratch;

[0007] Limited simulation efficiency: Traditional methods struggle to meet performance requirements for multi-field coupling, large-scale simulation, or real-time reasoning tasks.

[0008] To solve the above problems, PDE approximation methods based on neural networks have gradually emerged in recent years, mainly including the following two categories:

[0009] 1. Physical information-driven approach: By embedding partial differential operator residuals, boundary conditions, etc. into the network loss, the network is guided to approach a solution that satisfies physical laws;

[0010] 2. Neural operator method based on function-to-function mapping: directly learn the mapping relationship between input functions (such as source terms, boundary conditions) and target solution functions to achieve efficient end-to-end prediction.

[0011] Neural operator methods have significant advantages such as no grid requirement, cross-task generalization, and support for batch solutions, making them an important research hotspot in current numerical computing and inversion modeling. However, existing neural operator methods still face the following core challenges:

[0012] Limited structural expression capabilities: Mainstream neural operator structures typically use a unified fixed transformation method to process spatial coordinates, lacking structural adaptability and making it difficult to maintain accuracy in variable boundaries, local singular fields, or complex geometric domains.

[0013] Simple fusion mechanism: Input functions and coordinate features are usually fused using linear weighting or simple inner product methods, which cannot fully model the high-order interactions and nonlinear synergies between the two.

[0014] Weak physical consistency: Most structures rely primarily on data fitting, without explicitly introducing the physical constraints of differential equations, which can easily lead to physically unreasonable "pseudo-solutions";

[0015] Unstable optimization path: In small sample or multi-task training scenarios, the gradient propagation path is long and oscillating, causing the model to easily fall into local optimality or overfitting.

[0016] Therefore, current research urgently needs to build a neural operator framework that has the following capabilities: · Dynamically adjust its structural expression according to spatial input;

[0017] Ability to establish a deep fusion mechanism between functions and coordinates;

[0018] Introducing physical consistency regularization to ensure the physical rationality of the solution;

[0019] And have a clear training phase plan to improve model controllability and convergence.

[0020] In response to the above problems, the present invention proposes a neural operator model with a structural regulation mechanism, a collaborative modulation path and a staged training process, which provides a new numerical solution path for PDE problems that is more efficient, stable and physically feasible. Summary of the Invention

[0021] Purpose of the present invention:

[0022] The present invention aims to overcome the problems of weak structural expression ability, single input-output fusion method, and insufficient physical consistency guidance in current neural operator methods. It proposes a neural operator modeling method that integrates structural regulation mechanism, multi-level interactive modulation path and staged training process, which is suitable for solving partial differential equations under complex boundaries and small sample conditions.

[0023] The method includes the following key structures and processes:

[0024] 1. Neural Operator Structure Design

[0025] The overall structure of the model consists of five modules:

[0026] 1. Input representation encoder

[0027] It is used to map and encode input functions (such as boundary conditions, source terms, or initial states) to generate feature representation vectors.

[0028] 2. Coordinate Feature Generator

[0029] It receives spatial coordinate input and outputs a set of structural adjustment coefficients to drive the nonlinear transformation of spatial expression.

[0030] 3.Structure control function library

[0031] The weight matrix is ​​composed of several adjustable functions, which can be initialized from a priori basis functions or randomly. During the training process, it partially participates in backpropagation through a staged activation mechanism. Its output constitutes a spatially dependent dynamic feature basis.

[0032] 4. Collaborative Modulation Fusion Module

[0033] This module uses a multi-level interaction mechanism to align and fuse the input encoding and coordinate representation in the feature dimension to form a joint representation. This fusion strategy does not rely on traditional inner product operations, but instead adopts a weighted integration method based on a structural adjustment path based on relationship weights.

[0034] 5. Demapping module

[0035] Responsible for inputting the fused high-dimensional features into the neural mapping network to generate an estimate of the PDE solution at the target coordinate.

[0036] 2. Training Strategy and Loss Function Design

[0037] Model training adopts a phased optimization process to improve stability and accuracy. It is divided into the following three stages:

[0038] 1. The structure is fixed during the preheating phase

[0039] In the initial stage, only the input encoder and coordinate transformer are trained, and the structure function library parameters remain static to ensure the stability of the early modeling.

[0040] 2. Self-supervised reconstruction phase

[0041] After converging to a certain error level, reconstruction loss is added to optimize the matching accuracy between the feature fusion path and the coordinate mapping.

[0042] 3. Physical consistency coordination stage

[0043] Enable weight updates of the structured function library and introduce residual loss calculated by automatic differentiation. The residual term is constructed based on the target PDE form, for example, for the Laplace equation, its form is:

[0044]

[0045] in is the model prediction solution, f(x) is the source term, Δ is the Laplace operator and the final loss function is as follows:

[0046]

[0047] in:

[0048] ·L data :Data driven error;

[0049] ·L physics : physical residual loss;

[0050] · Structured function library regular terms

[0051] λ, β: adjustment factors

[0052] 3. Scope of application and performance advantages

[0053] This method is widely applicable to solving the following types of partial differential equations:

[0054] Elliptic equations: such as variable coefficient Poisson equation and Darcy flow;

[0055] Parabolic equations: such as the time-dependent heat conduction equation;

[0056] Multi-field coupled systems: such as nonlinear diffusion-convection-reaction models.

[0057] Compared with traditional neural operator structures, this invention has the following advantages:

[0058] Structural adaptive adjustment: significantly enhances the model's ability to express spatial heterogeneity and local changes;

[0059] The fusion path is controllable: a multi-level modulation mechanism is used to improve the modeling accuracy between input and output;

[0060] Explicit guidance of physical constraints: Improves the physical understanding and generalization ability of the solution through residual penalty; Stronger training convergence stability: The staged optimization process effectively reduces overfitting and gradient oscillation. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 Schematic diagram of the overall network structure of the model of the present invention;

[0062] Figure 2 This is a flow chart of the model of the present invention during the training phase;

[0063] Figure 3 This is an explanation of the model structure generation process in the present invention;

[0064] Figure 4 This is a description of the internal structure of the collaborative modulation fusion module in the present invention; DETAILED DESCRIPTION

[0065] The neural operator modeling method proposed in the present invention is described in detail below with reference to the accompanying drawings. It should be noted that the following embodiments are only used to illustrate the technical solution of the present invention and do not constitute a limitation on the scope of protection of the present invention.

[0066] 1. Overall network structure (see Figure 1 )

[0067] like Figure 1 As shown, the neural operator structure of the present invention mainly includes the following five modules:

[0068] 1. Input Encoder: This module extracts information about the input function, such as initial conditions, boundary functions, and source terms, and encodes it into a high-dimensional feature representation. This module can be composed of several fully connected layers, including nonlinear activation and normalization operations, and supports multiple input modal combinations.

[0069] 2. Coordinate Transformer: This module receives spatial coordinate information x and outputs a set of structure adjustment coefficient vectors to drive the generation of spatial structure representations. This module can be designed as a shallow or deep neural network, with the output vector dimension matching the structure adjustment function library.

[0070] 3. Structure Modulation Basis: This library constructs an initial set of function basis matrices of size [m, d], where mmm represents the structural dimension and ddd represents the spatial dimension. Initially, the library parameters remain static and are gradually activated for optimization after the model has stabilized, improving structural adaptability.

[0071] 4. Cross-Modulation Fusion: This module fuses input features with spatial structure representations on the feature dimension. This fusion utilizes a multi-level interaction mechanism, bypassing traditional inner-product attention and instead achieving feature modulation through relation-guided weighting. This module includes a structural modulation gate unit and a projection-merging pipeline.

[0072] 5. Output Predictor: This module performs nonlinear mapping on the fused high-dimensional representation and outputs the predicted solution of the target PDE at a specific coordinate point. It can be composed of several layers of MLPs and can output scalar or tensor solutions.

[0073] 2. Training Phase Process (See Figure 2 )

[0074] like Figure 2 As shown, the present invention adopts a three-stage training process:

[0075] Phase 1 (Structural preheating phase)

[0076] Freeze structure control function library parameters;

[0077] Only train the input encoder and coordinate transformer;

[0078] The loss term is the data error term L data

[0079] Phase 2 (Fusion and Alignment Phase)

[0080] Activate some weights of the structure function library;

[0081] Introducing reconstruction loss L recon , improve the quality of feature representation;

[0082] Jointly optimize the fusion path.

[0083]

[0084] Phase 3 (Physical Consistency Phase)

[0085] ·Introduce the PDE residual loss term L obtained by automatic differentiation physics

[0086] Synchronously optimize the entire model to improve physical consistency;

[0087] The final loss function is:

[0088] L total =L data +λL physics +β||Φ||2

[0089] Among them, Φ is the parameter set of the structure control function library, and λ and β are the loss weight hyperparameters.

[0090] 3. Description of the structure generation process (see Figure 3 )

[0091] like Figure 3 As shown in Figure 2, the generation process of the structural control function is as follows:

[0092] The coordinate input x first generates the structure weight through CoordinateTransformer

[0093] vector;

[0094] This vector is input into the StructureModulationBasis module as the adjustment coefficient.

[0095] Perform affine combination with function library to generate dynamic spatial structure expression;

[0096] The resulting structure vector is then collaboratively modeled with the input features (from InputEncoder) in the Cross-ModulationFusion module.

[0097] Finally, a unified and fused high-dimensional feature vector is formed and input into the demapping module to generate the predicted value.

[0098] This design supports spatial heterogeneity modeling and boundary adaptability while maintaining the overall trainability of the network.

[0099] 4. Description of the internal structure of the coordinated modulation fusion module (see Figure 4 )

[0100] like Figure 4 As shown, the Cross-Modulation Fusion module of the present invention mainly consists of three sub-units:

[0101] RelationEncoder

[0102] This unit receives two inputs: the function feature vector from the input encoder and the spatial expression vector output by the structure control module;

[0103] A set of fused guidance vectors is generated through fully connected mapping and nonlinear activation to characterize the semantic relationship between the two.

[0104] ModulationGate

[0105] Generate a set of dynamic fusion weights based on the relationship guidance vector;

[0106] The weight acts on the input features to enhance or suppress specific dimensions.

[0107] The adjustment form is:

[0108] f mod =σ(W1r+b1)⊙f input

[0109] Where r is the relationship vector, σ is the Sigmoid function, and ⊙ represents element-by-element multiplication.

[0110] Fusion output layer (ResidualMerge)

[0111] The modulated features are superimposed on the structural features through residual connections and fed into the next module;

[0112] This connection method improves the information integrity and gradient stability of the fusion path.

[0113] The entire collaborative fusion process does not rely on the QKV structure in the traditional attention mechanism, but performs feature enhancement through a relationship-driven gating method, effectively avoiding high-overlap design and enhancing the structural expression's ability to perceive the input function.

[0114] V. Examples of applicable issues

[0115] This method can be used to solve two-dimensional and three-dimensional steady-state and unsteady partial differential equations, and is applicable to the following representative scenarios: elliptic equations (such as Poisson equations and Darcy flow);

[0116] Darcy flow model: elliptic equations with variable coefficient permeability field; bidirectional coupling problems (such as nonlinear diffusion);

[0117] Nonlinear diffusion problems: such as the reaction-diffusion-convection coupled model; this model has the following advantages:

[0118] Heat conduction problems on variable geometry domains;

[0119] Multi-physics coupled inversion problems.

Claims

1. A structure-regulated neural operator modeling method, characterized in that: Includes the following modules: (1) Conditional encoding module: used to extract information about boundary conditions, initial conditions, or source functions and generate semantic feature representations; (2) Spatial mapping module: used to receive spatial coordinate information and output a set of structural adjustment coefficients; (3) Basis function control unit: contains a set of adjustable configuration vector libraries, which are used to selectively activate different spatial response modes through structural adjustment coefficients; (4) Co-modulation module: Constructs a deep fusion representation based on the synergy between input semantic features and spatial response modalities; (5) Solution mapping module: used to perform nonlinear mapping on the fusion representation and output the solution function value of the partial differential equation.

2. The method according to claim 1, wherein: The structure control function library is an [m, d]-dimensional real number matrix, where m is the number of control functions and d is the spatial dimension; the matrix remains static in the early stage of training and is enabled to update weights after reaching an error threshold.

3. The method according to claim 1, wherein: The coordinate transformation generator outputs a structural weight vector through a multi-layer mapping function, and the vector is affine transformed with the function library matrix to output a coordinate representation result.

4. The method according to claim 1, wherein: The training process consists of three stages: (1) Initial stage: freeze the control vector and only train the mapping module; (2) Structural adaptation stage: enabling joint optimization of configuration vectors and introducing input self-reconstruction objectives; (3) Physical constraint stage: Add the partial differential residual generated by automatic differentiation to construct the physical consistency loss.

5. The method according to claim 1, wherein: The training of the neural operator model is divided into multiple stages, including at least: During the structural adaptation warm-up phase, only the input encoder and coordinate converter are updated; In the collaborative optimization phase, the structural function library parameters are updated simultaneously and physical consistency loss is introduced.

6. The training method according to claim 5, characterized in that: The physical residual loss term calculates the second-order derivative of the coordinates through automatic differentiation to construct a residual term consistent with the target PDE, for example: in is the model prediction solution, L PDE It is the physical residual loss term, which is used to measure the physical consistency of the prediction results to the target PDE. The residual calculation is based on the automatic differentiation technology.

7. The training method according to claim 5, characterized in that: The training objective function consists of three parts: Data error loss; Physical residual loss structure constraint regularization term; Structural constraint regularization term, which is used to suppress the parameter amplitude fluctuation in the function library.

8. The neural operator model according to claim 1, characterized in that: Both the input representation encoder and the coordinate transformer adopt a composable neural mapping structure, and their nonlinear units use Swish or GELU functions.

9. The neural operator model according to claim 1, characterized in that: The method is applicable to two-dimensional or three-dimensional steady-state or transient partial differential equation problems, including but not limited to nonlinear diffusion equations, Poisson equations with source terms, and variable coefficient Darcy problems.

Images