A method for solving mantle flow velocity based on continuous learning and physical information network

Abstract

The present invention discloses a method for solving mantle flow velocity based on continuous learning and physical information network, which belongs to the field of machine learning technology and includes the steps of: determining the mantle convection equation group to be solved, boundary conditions and initial conditions, wherein the mantle convection equation group includes 5 equations ~; generating a training data set; establishing a neural network and training it to convergence to obtain a mantle flow velocity solution model, which is used to solve any sample point to be tested in the spatial region and time region under study. During the training process of the neural network, a loss function ~ is defined, and continuous learning technology is used to ~optimize ~ in sequence. The present invention can reduce the computational burden of high-dimensional problems and optimize computing resource requirements; improve training stability and reduce convergence difficulty; ensure that the neural network learns the physical properties of the equation group; support transfer learning and improve knowledge transfer between tasks.

Description

Technical Field

[0001] The present invention relates to the technical field of machine learning, and in particular to a method for solving mantle flow velocity based on continuous learning and physical information network. Background Art

[0002] Mantle convection refers to the movement of mantle materials and heat conduction caused by thermal differences inside the mantle. Mantle convection plays an important role in the heat flow and material circulation inside the earth. It is one of the main mechanisms of energy transmission inside the earth and an important driving force for the movement of earth plates and seismic activity. It has a significant impact on geodynamics and the evolution of the earth's surface. Based on geophysics, fluid mechanics and thermodynamics, a set of partial differential equations is established to simulate the movement, temperature and material transport of mantle convection, revealing the laws of change of these physical quantities in space and time. The results can provide a detailed understanding of mantle convection and reveal the relationship between mantle convection and earth phenomena such as plate tectonics and volcanic activity.

[0003] The solution of the mantle convection equations is one of the core concerns of the research work. Traditional solution methods are generally divided into two categories: analytical solution and numerical solution. The analytical solution obtains the solution of the equation through mathematical derivation, while the numerical methods such as finite difference method and finite element method discretize the space and time variables, transform the partial differential equation into algebraic equations, and then obtain the approximate solution through numerical calculation. However, when the equations involve the coupling of multiple space dimensions and time dimensions, the complexity increases significantly, the derivation of analytical solutions will be very difficult, and the existence cannot be guaranteed. With the increase of dimensions, the scale of numerical calculations expands rapidly, and the amount of calculation and storage requirements rise sharply. Solving may require huge computing resources and time. Irregular boundary conditions and initial conditions will also increase the difficulty of analytical derivation and numerical calculation. In actual engineering, it is usually necessary to complete high-precision simulations with limited computing resources. Therefore, how to optimize the calculation process and reduce memory consumption is always a problem to be solved.

[0004] In order to effectively deal with the challenges brought by factors such as high dimensions, complex initial conditions and boundary conditions, and large problem scale in solving equations, researchers have begun to combine emerging technologies such as machine learning and deep learning to improve computational efficiency and accuracy. Currently, the commonly used methods are as follows:

[0005] (1) Physics-Informed Neural Networks (PINN) integrate multiple equations, boundary conditions, initial conditions and other information in the system of equations into the loss function in a unified way, and use the powerful fitting ability of neural networks to minimize the loss function to optimize the network weights. During the training process, the system of partial differential equations is directly solved, while ensuring that the network output meets the physical constraints of the system of equations. This method can not only handle the problem of multi-equation coupling, but also avoid the strong dependence of traditional numerical methods on discretization accuracy to a certain extent, so that PINN can effectively use physical constraints in the absence of data and ensure the physical consistency of the solution. When dealing with a single partial differential equation, PINN fully demonstrates the advantages and potential of neural networks in the field of partial differential equation solving, but it may still face the following problems when dealing with a system of partial differential equations in the context of multiple equation coupling:

[0006] (1.1) Difficulty in optimization and unstable numerical solutions

[0007] When traditional PINN solves a system of partial differential equations, considering the coupling form of the system of equations, PINN usually combines the residuals of all equations into a unified loss function and performs joint optimization. However, due to the different properties of the equations, this approach may cause the residuals of some equations to dominate the optimization process, resulting in instability in the training process of the neural network, or the solutions of some equations are masked by the strong constraints of other equations and cannot converge to high-precision solutions. At the same time, the gradient conflict problem between different equations will also affect the overall convergence effect of the system of equations. As the complexity of the equations increases during the training process, the neural network may not be able to balance the constraints of each equation well, resulting in numerical instability or incomplete convergence.

[0008] (1.2) High computing resource requirements

[0009] Since the multi-equation coupling problem involves higher-dimensional calculations and more network parameters, fitting multiple coupled equations simultaneously requires the network to have greater expressive power. Especially when the equation set is large, a single network may be difficult to handle. In particular, when solving large-scale, high-dimensional equation sets, the training process may require a lot of computing resources and a long training time.

[0010] (2) Deep Galerkin Method (DGM) is a method for solving partial differential equations that combines deep learning with classical numerical methods. Unlike traditional numerical methods, DGM does not require explicit discretization of space and time variables, nor does it rely on complex grid division or discretization calculations. It combines neural networks with the Galerkin method and can solve partial differential equations without a grid. It is particularly suitable for complex nonlinear problems that are difficult to solve with traditional numerical methods. The basic idea of ​​this method is similar to PINN. Both methods represent the solution of partial differential equations as the output of a neural network. The difference between the training output of the neural network and the residual of the partial differential equation is used to optimize the parameters of the neural network by minimizing the loss function so that its output solution satisfies the partial differential equation and its initial and boundary conditions at the same time. Unlike the PINN method, the core of DGM is to transform PDE into a weak form for processing. The traditional Galerkin method transforms the strong form of PDE into a weak form so that it can be solved in a finite-dimensional space. DGM calculates the residual of the output solution of the neural network and optimizes the weights of the neural network by embedding the residual into the loss function so that the output solution of the network meets the weak form of PDE. The core idea of ​​the Deep Galerkin method comes from the Galerkin method, but it still does not effectively solve the following problems:

[0011] (2.1) High-dimensional problems consume large amounts of computational resources

[0012] Although DGM has certain advantages over traditional numerical methods in high-dimensional problems, the training process of neural networks is still very computationally intensive. In high-dimensional space, as the number of sample points increases, the demand for training data and computing resources will increase rapidly, making training very slow and time-consuming.

[0013] (2.2) Instability of the training process

[0014] The training process of neural networks may face problems such as local minima and saddle points. Especially in complex PDE or multi-equation coupling problems, the training may become unstable and difficult to converge. For some nonlinear equations that are difficult to solve, the optimization algorithm may fall into an unstable area, resulting in the inability to guarantee the quality of the solution.

[0015] (2.3) Multi-equation coupling problem

[0016] In a multi-equation coupled PDE system, DGM needs to combine the residuals and boundary conditions of all equations into one loss function. However, the mutual influence between different equations may make the training process complicated and unstable, especially when the system of equations involves strong coupling, optimization may become more difficult.

[0017] Glossary:

[0018] Continuous learning: Also known as lifelong learning, it is an important research direction in the field of machine learning. It aims to enable the model to learn new tasks while maintaining the memory and ability to solve previous tasks. Different from traditional joint learning, continuous learning focuses on how the model can avoid forgetting the knowledge it has learned when facing dynamically changing learning tasks and can efficiently handle new tasks.

[0019] One of the core challenges of continuous learning technology is the problem of catastrophic forgetting. Catastrophic forgetting refers to the fact that when learning a new task, the model tends to lose its memory and performance of previous tasks, especially when there is no adequate memory retention mechanism. To address this problem, continuous learning methods have adopted many different strategies, such as retaining important knowledge through regularization techniques, or avoiding interference between tasks by assigning independent network modules to each task. In addition, methods such as transfer learning and multi-task learning are also commonly used in continuous learning to help models make full use of previous experience when handling new tasks and avoid learning from scratch.

[0020] Elastic Weight Consolidation (EWC) is a classic continuous learning method that keeps the memory of important parameters of previous tasks by introducing regularization terms. During training, the loss function of EWC is divided into two parts, one is the loss of the current task, and the other is the regularization term. For each network parameter, EWC will calculate the Fisher information matrix of the parameter to measure the importance of the parameter in the previous task. The larger the value, the more important the parameter. During training, this method will penalize weight changes that may cause performance degradation in previous tasks.

[0021] In practical applications, continuous learning is widely used to handle a range of tasks and dynamic changes in data. For example, in the field of natural language processing, continuous learning enables the model to always maintain efficient adaptability when facing different contexts and language tasks; in the field of robotics, continuous learning can help robots perform tasks in a constantly changing environment and optimize their action strategies over time. The technology of continuous learning can significantly improve the flexibility of the system, taking into account multi-task learning while improving the generalization ability of the model. Summary of the invention

[0022] The purpose of the present invention is to provide a method for solving the mantle flow velocity based on continuous learning and physical information networks, which can solve the above-mentioned problems of numerical instability, large consumption of computing resources in high-dimensional problems, unstable training process, and multi-equation coupling.

[0023] In order to achieve the above object, the technical solution adopted by the present invention is as follows: a method for solving mantle flow velocity based on continuous learning and physical information network, comprising the following steps;

[0024] S1, determine the mantle convection equations to be solved, boundary conditions and initial conditions; the mantle convection equations include 5 equations ~ , where the mth equation is , 1≤m≤5;

[0025] S2, generating a training data set T, including steps S21 to S23;

[0026] S21, determine the space region, time region B, and sample point s for the study, the space region includes the core-mantle region , surface and core-mantle boundary region , , is the coordinate value of the sample point s on the i-th axis of the three-dimensional coordinate system at time t, i=1,2,3;

[0027] S22, in Get multiple sample points from B to form a point set in the domain ,exist Get multiple sample points from B to form a boundary point set ,from and Select multiple sample points from the set, set t=0, and form the initial point set ;

[0028] S23, will , , Merge into training data set T;

[0029] S3, establish a neural network, whose network parameters are vectors , and taking sample point s as input, the function group corresponding to sample point s is the expected output, where is the velocity component of the i-th axis at the sample point s at time t, w and p are the mantle temperature and hydrostatic pressure of the sample point s at time t, respectively;

[0030] S4, train the neural network with the training data set T to optimize , including steps S41 to S44;

[0031] S41, define the control residual loss term , boundary condition residual , initial condition residual ;

[0032] , , ;

[0033] In the formula, , , They are , , The number of sample points in is the square of the L2 norm, , , When the neural network inputs sample points s, , , The corresponding predicted value is for exist The speed value on is the average velocity of the mantle flow along the i-th axis of the three-dimensional coordinate system;

[0034] S42, let m=i=1, generate optimization The loss function , ,in , , They are , , The weight factor of

[0035] S43, input the training data set T into the neural network to minimize Adjust the neural network parameters to get the optimized network parameter vector ;

[0036] S44, calculation The importance of each component in The importance of ;

[0037] S5, use the training data set T to train the neural network optimization ~ , including steps S51 to S53;

[0038] S51, let m=m+1, if m≥6, then end;

[0039] S52, if m=2 or 3, let i=m, and generate the optimization according to the following formula The loss function ;

[0040] ,

[0041] ,

[0042] In the formula, is the regularization loss term, for The weight factor of is the weight adjustment factor, for The lth component in

[0043] If m=4, the optimization is generated according to the following formula The loss function ;

[0044] ;

[0045] ,

[0046] ,

[0047] In the formula, , They are The corresponding boundary condition residuals and initial condition residuals are , They are The mantle temperature and heat flow on is the average temperature of the mantle;

[0048] If m=5, generate optimization The loss function , ;

[0049] S53, according to ~ Train the neural network sequentially to optimize ~ ;

[0050] S6, repeat S4-S5 until the neural network converges to obtain the mantle flow velocity solution model;

[0051] S7, obtain any sample point to be measured in the studied spatial and temporal regions, input the mantle flow velocity solution model, and output the velocity of the sample point to be measured. , , , .

[0052] Preferably, the mantle convection equations are:

[0053] ,

[0054] Among them, for sample point s, is the initial state density of the mantle, , , The components of the body force acting on the sample point s on the 1st axis, 2nd axis, and 3rd axis respectively, is the volume expansion coefficient of the mantle, is the kinematic viscosity coefficient, is the thermal diffusivity of the mantle, is the average mantle temperature, is the Laplace operator.

[0055] As a preference: the initial conditions are: , ;

[0056] The boundary conditions are: , , , ;

[0057] In the boundary conditions, For p The pressure value on.

[0058] As a preferred embodiment: in S22, the method of obtaining sample points is to uniformly sample in the spatial region and the time region B.

[0059] As a preference: When training a neural network, a hybrid optimization strategy is used to optimize the loss function by combining the Adam optimizer and the L-BFGS optimizer.

[0060] Preferably, the neural network is a feedforward neural network, comprising an input layer, a hidden layer and an output layer.

[0061] As a preferred embodiment: in S44, According to the following formula, .

[0062] In the present invention, different equations use different loss functions. For example: To control the residual loss term, All sample points in the are involved in the calculation and are used to constrain the model to follow the boundary conditions; is the initial condition residual, only Internal sample points participate in the calculation and are used to constrain the model to follow the initial conditions. The second-order derivative is used to calculate the network parameter vector used to evaluate the neural network The importance of each component in , thus generating a regularized loss term .

[0063] Compared with the prior art, the advantages of the present invention are:

[0064] (1) Reduce the computational burden of high-dimensional problems and optimize computing resource requirements.

[0065] In high-dimensional complex systems, the computational complexity of traditional numerical methods often increases with the increase of spatial dimensions and the number of equations. Especially when dealing with high-dimensional complex system problems, traditional methods require a lot of computing resources and storage space, and may even face the curse of dimensionality. Training equations sequentially can reduce the computational burden of each training. Compared with training the entire set of equations at one time, gradually training each equation means that the goal of each optimization is simpler, the computing requirements are dispersed, and the complexity of processing huge networks in high-dimensional space is avoided. In addition, step-by-step training helps to dynamically adjust the capacity requirements of the network without setting computing resources in advance, which improves the efficiency of computing resource allocation.

[0066] (2) Improve training stability and reduce convergence difficulty.

[0067] In a system of partial differential equations, equations are often coupled to each other through different physical quantities. Traditional methods often require the network to optimize the residual loss of all equations at the same time, which makes the optimization process too complicated, difficult to converge, and even unstable. The method proposed in the present invention decomposes the complex multi-equation coupling problem into multiple simpler subtasks, optimizing one equation at a time, so that the network can be more focused when processing each equation, avoiding the gradient conflict or gradient competition problems that may exist between different equations during training. The regularization loss term generated based on the EWC method further constrains the changes in network parameters, reduces the occurrence of drastic changes in parameters during multi-equation training, and improves training stability.

[0068] (3) Ensure that the neural network learns the physical properties of the equations.

[0069] Only one equation is optimized during each training, so that the physical constraints of each equation can be fully learned, avoiding a strong constraint equation from having too much impact on the training of other equations. In addition, EWC constrains the parameters of the neural network to ensure that the network does not lose the knowledge of other equations that have been learned when training one equation, thereby ensuring that the neural network expresses the physical properties of all equations.

[0070] (4) Support transfer learning and improve knowledge transfer between tasks.

[0071] In multi-task learning or multi-equation coupling problems, some equations may have certain similarities or regularities with other equations, and traditional methods often ignore such similarities. Using EWC's knowledge transfer, the method can share the knowledge learned from the learned equations when training new equations, which means that subsequent equations can utilize the previous training results to a certain extent, thereby accelerating the training process and improving the performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 A flowchart of training a neural network according to the present invention;

[0073] Figure 2 This is a diagram of the neural network structure. DETAILED DESCRIPTION

[0074] The present invention will be further described below in conjunction with embodiments and drawings.

[0075] Example 1: See Figure 1 and Figure 2 , a method for solving mantle flow velocity based on continuous learning and physical information network, comprising the following steps;

[0076] S1, determine the mantle convection equations to be solved, boundary conditions and initial conditions; the mantle convection equations include 5 equations ~ , where the mth equation is , 1≤m≤5;

[0077] S2, generating a training data set T, including steps S21 to S23;

[0078] S21, determine the space region, time region B, and sample point s for the study, the space region includes the core-mantle region , surface and core-mantle boundary region , , is the coordinate value of the sample point s on the i-th axis of the three-dimensional coordinate system at time t, i=1,2,3;

[0079] S22, in Get multiple sample points from B to form a point set in the domain ,exist Get multiple sample points from B to form a boundary point set ,from and Select multiple sample points from the set, set t=0, and form the initial point set ;

[0080] S23, will , , Merge into training data set T;

[0081] S3, establish a neural network, whose network parameters are vectors , and taking sample point s as input, the function group corresponding to sample point s is the expected output, where is the velocity component of the i-th axis at the sample point s at time t, w and p are the mantle temperature and hydrostatic pressure of the sample point s at time t, respectively;

[0082] S4, train the neural network with the training data set T to optimize , including steps S41 to S44;

[0083] S41, define the control residual loss term , boundary condition residual , initial condition residual ;

[0084] , , ;

[0085] In the formula, , , They are , , The number of sample points in is the square of the L2 norm, , , When the neural network inputs sample points s, , , The corresponding predicted value is for exist The speed value on is the average velocity of the mantle flow along the i-th axis of the three-dimensional coordinate system;

[0086] S42, let m=i=1, generate optimization The loss function , ,in , , They are , , The weight factor of

[0087] S43, input the training data set T into the neural network to minimize Adjust the neural network parameters to get the optimized network parameter vector ;

[0088] S44, calculation The importance of each component in The importance of ;

[0089] S5, use the training data set T to train the neural network optimization ~ , including steps S51 to S53;

[0090] S51, let m=m+1, if m≥6, then end;

[0091] S52, if m=2 or 3, let i=m, and generate the optimization according to the following formula The loss function ;

[0092] ,

[0093] ,

[0094] In the formula, is the regularization loss term, for The weight factor of is the weight adjustment factor, for The lth component in

[0095] If m=4, the optimization is generated according to the following formula The loss function ;

[0096] ;

[0097] ,

[0098] ,

[0099] In the formula, , They are The corresponding boundary condition residuals and initial condition residuals are , They are The mantle temperature and heat flow on is the average temperature of the mantle;

[0100] If m=5, generate optimization The loss function , ;

[0101] S53, according to ~ Train the neural network sequentially to optimize ~ ;

[0102] S6, repeat S4-S5 until the neural network converges to obtain the mantle flow velocity solution model;

[0103] S7, obtain any sample point to be measured in the studied spatial and temporal regions, input the mantle flow velocity solution model, and output the velocity of the sample point to be measured. , , , .

[0104] In the present invention, the mantle convection equations are:

[0105] ,

[0106] Among them, for sample point s, is the initial state density of the mantle, , , The components of the body force acting on the sample point s on the 1st axis, 2nd axis, and 3rd axis respectively, is the volume expansion coefficient of the mantle, is the kinematic viscosity coefficient, is the thermal diffusivity of the mantle, is the average mantle temperature, is the Laplace operator.

[0107] The initial conditions are: , ;

[0108] The boundary conditions are: , , , , in the boundary conditions, For p The pressure value on.

[0109] In S22, the method of obtaining sample points is uniform sampling in the spatial region and the time region B. When training the neural network, a hybrid optimization strategy is used to optimize the loss function by combining the Adam optimizer and the L-BFGS optimizer. The neural network is a feedforward neural network, including an input layer, a hidden layer and an output layer. In S44, .

[0110] The mantle convection equations to be solved contain 5 equations: ~ , used to solve the function group The present invention designs a neural network for inputting sample points When , predict and output the corresponding sample point . The values ​​in are the true values ​​of the sample points, which are used to train the neural network. The value in is the predicted value output by the neural network, which is used to predict the sample points to be tested. In addition, other parameters in the mantle convection equations such as etc. are all known quantities.

[0111] Regarding the training of neural networks, for 5 equations ~ , the present invention designs different loss functions according to the actual situation ~ The five equations are considered as five tasks and the continuous learning technology is used to optimize the five equations in sequence. Since the continuous learning technology avoids catastrophic forgetting, the five equations have smaller errors in the final model.

[0112] about , according to the formula In this formula, for , The sampling points in the t are deleted or t=0, and the initial points are formed. .

[0113] Regarding the sample point s , is the coordinate value of the sample point s on the i-th axis of the three-dimensional coordinate system at time t. This is because the three-dimensional coordinate system includes the x-axis, y-axis, and z-axis. When i is 1, 2, and 3, it corresponds to the x-axis, y-axis, and z-axis, respectively.

[0114] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for calculating mantle flow velocity based on continuous learning and physical information network, characterized by: The steps include: S1, determine the mantle convection equations to be solved, boundary conditions and initial conditions; the mantle convection equations include 5 equations ~ , where the mth equation is , 1≤m≤5; S2, generating a training data set T, including steps S21 to S23; S21, determine the space region, time region B, and sample point s for the study, the space region includes the core-mantle region , surface and core-mantle boundary region , , is the coordinate value of the sample point s on the i-th axis of the three-dimensional coordinate system at time t, i=1,2,3; S22, in Get multiple sample points from B to form a point set in the domain ,exist Get multiple sample points from B to form a boundary point set ,from and Select multiple sample points from the set, set t=0, and form the initial point set ; S23, will , , Merge into training data set T; S3, establish a neural network, whose network parameters are vectors , and taking sample point s as input, the function group corresponding to sample point s is the expected output, where is the velocity component of the i-th axis at the sample point s at time t, w and p are the mantle temperature and hydrostatic pressure of the sample point s at time t, respectively; S4, train the neural network with the training data set T to optimize , including steps S41 to S44; S41, define the control residual loss term , boundary condition residual , initial condition residual ; ， ， ； In the formula, , , They are , , The number of sample points in is the square of the L2 norm, , , When the neural network inputs sample points s, , , The corresponding predicted value is for exist The speed value on is the average velocity of the mantle flow along the i-th axis of the three-dimensional coordinate system; S42, let m=i=1, generate optimization The loss function , ,in , , They are , , The weight factor of S43, input the training data set T into the neural network to minimize Adjust the neural network parameters to get the optimized network parameter vector ; S44, calculation The importance of each component in The importance of ; S5, use the training data set T to train the neural network optimization ~ , including steps S51 to S53; S51, let m=m+1, if m≥6, then end; S52, if m=2 or 3, let i=m, and generate the optimization according to the following formula The loss function ; ， ， In the formula, is the regularization loss term, for The weight factor of is the weight adjustment factor, for The lth component in If m=4, the optimization is generated according to the following formula The loss function ; ； ， ， In the formula, , They are The corresponding boundary condition residuals and initial condition residuals are , They are The mantle temperature and heat flow on is the average temperature of the mantle; If m=5, generate optimization The loss function , ; S53, according to ~ Train the neural network sequentially to optimize ~ ; S6, repeat S4-S5 until the neural network converges to obtain the mantle flow velocity solution model; S7, obtain any sample point to be measured in the studied spatial and temporal regions, input the mantle flow velocity solution model, and output the velocity of the sample point to be measured. , , , .

2. The method for calculating mantle flow velocity based on continuous learning and physical information network according to claim 1, characterized in that: The mantle convection equations are: ， Among them, for sample point s, is the initial state density of the mantle, , , The components of the body force acting on the sample point s on the 1st axis, 2nd axis, and 3rd axis respectively, is the volume expansion coefficient of the mantle, is the kinematic viscosity coefficient, is the thermal diffusivity of the mantle, is the average mantle temperature, is the Laplace operator.

3. The method for calculating mantle flow velocity based on continuous learning and physical information network according to claim 2, characterized in that: The initial conditions are: , , i=1,2,3; The boundary conditions are: , , , ; In the boundary conditions, For p The pressure value on.

4. The method for calculating mantle flow velocity based on continuous learning and physical information network according to claim 1, characterized in that: In S22, the method of obtaining sample points is to uniformly sample in the spatial region and the time region B.

5. The method for calculating mantle flow velocity based on continuous learning and physical information network according to claim 1, characterized in that: When training neural networks, a hybrid optimization strategy is used to optimize the loss function by combining the Adam optimizer and the L-BFGS optimizer.

6. The method for calculating mantle flow velocity based on continuous learning and physical information network according to claim 1, characterized in that: The neural network is a feedforward neural network, comprising an input layer, a hidden layer and an output layer.

7. The method for calculating mantle flow velocity based on continuous learning and physical information network according to claim 1, characterized in that: In S44, According to the following formula: 。

Images