Uncertainty quantification method based on embedded physical information neural network

Abstract

The invention discloses an uncertainty quantification method based on an embedded physical information neural network. The problems that data uncertainty cannot be eliminated and a model uncertainty method is difficult to select and popularize are solved. According to the scheme, the method comprises the following steps: randomly selecting an input-output sample containing data noise, establishing and training a data-driven neural network model, and establishing and training a neural network model embedded with physical information in combination with a physical equation satisfied by an input-output variable; based on the neural network model driven by the data and the neural network model embedded with the physical information, predicting the same sample, and comparing prediction results of the two neural network models; based on a neural network model embedded with physical information, a Bayesian method, a deep integration method and an MC Dropout method are used to improve the neural network model; and on the basis of the improved neural network model embedded with the physical information, predicting the same data, comparing the variance and the deviation of prediction results, and providing guidance for the selection of a model uncertainty quantification method.

Description

Technical Field

[0001] The present invention belongs to the interdisciplinary technical field of machine learning and uncertainty quantification, and mainly relates to an uncertainty quantification method based on machine learning. Background Art

[0002] With the development of computer science and technology, machine learning methods represented by neural networks have been increasingly applied to the modeling, analysis and control processes of physical science. In these fields, there are a large number of uncertain factors. In order to ensure the credibility of the prediction results of machine learning models, uncertainty quantification has become a research hotspot in recent years. Uncertainty in engineering is mainly divided into two categories: accidental uncertainty from the data itself and cognitive uncertainty caused by insufficient cognition of machine learning models. Research on these two types of uncertainty has become mature, but there are still two key technical problems: first, data uncertainty is an inherent attribute of the data itself, which cannot be eliminated with the increase of data volume, and the existing technology cannot reduce or eliminate the impact of data uncertainty; second, for model uncertainty quantification, current research methods are usually presented under specific technical tasks, which are difficult to extend to other task fields, and the application is limited. There is a lack of comparative research on model uncertainty quantification methods, and it is impossible to give the optimal technical method selection. In summary, in order to solve the technical problems that the impact of data uncertainty cannot be eliminated and the model uncertainty method is difficult to select and promote, the present invention proposes an uncertainty quantification method based on a neural network embedded with physical information. Summary of the invention

[0003] In order to overcome the shortcomings of the prior art and solve the technical problems that data uncertainty cannot be eliminated and model uncertainty methods are difficult to select and promote, the present invention provides an uncertainty quantification method based on a neural network embedded with physical information.

[0004] An uncertainty quantification method based on a neural network embedded with physical information includes a neural network model embedded with physical information and a model uncertainty quantification selection method based on improving the neural network model embedded with physical information; the steps of constructing the neural network model embedded with physical information are:

[0005] S1: Establish the input-output functional relationship of the model or system to be evaluated based on the differential equation, and calculate the noise-containing response output of the input-output functional relationship based on the differential equation according to the random distribution characteristics of the input variables of the model or system to be evaluated;

[0006] S2: Based on the input-output functional relationship of the differential equation, the noisy input-output samples are established to build and train the data-driven neural network model;

[0007] S3: Establish and train a neural network model embedded with physical information based on the input-output functional relationship, initial conditions and boundary conditions of the model or system to be evaluated by differential equations;

[0008] The steps of the model uncertainty quantization selection method based on the neural network model embedded with physical information are as follows:

[0009] S4: Based on the established neural network model embedded with physical information, the Bayesian method, the deep integration method and the MC Dropout method are used to improve it, and the neural network model embedded with physical information improved by the Bayesian method, the neural network model embedded with physical information improved by the deep integration method and the neural network model embedded with physical information improved by the MC Dropout are obtained respectively;

[0010] S5: For the neural network model embedded with physical information improved by the Bayesian method, the neural network model embedded with physical information improved by the deep integration method, and the neural network model embedded with physical information improved by MC Dropout, the same data set is used for prediction, and the prediction results of the neural network model embedded with physical information improved by the Bayesian method, the prediction results of the neural network model embedded with physical information improved by the deep integration method, and the prediction results of the neural network model embedded with physical information improved by MC Dropout are obtained respectively;

[0011] The prediction results of the neural network model embedded with physical information improved based on the Bayesian method, the prediction results of the neural network model embedded with physical information improved based on the deep integration method, and the prediction results of the neural network model embedded with physical information improved based on MC Dropout all include mean, variance, deviation and training time;

[0012] S6: Based on the prediction results of the neural network model embedded with physical information improved by the Bayesian method, the prediction results of the neural network model embedded with physical information improved by the deep integration method, and the prediction results of the neural network model embedded with physical information improved by MC Dropout, according to engineering requirements and selection strategies, the improved neural network model embedded with physical information is selected as the optimal neural network model embedded with physical information for model uncertainty quantification.

[0013] Furthermore, the specific steps of calculating the output response of the input-output functional relationship based on the differential equation containing noise are:

[0014] For the model or system to be evaluated, the input of the model or system to be evaluated is x, the output of the model or system to be evaluated is y, and the functional relationship between input and output is y=u(x), which is governed by the following differential equation:

[0015]

[0016] In the formula, represents the differential operator, f(x) is a function of known form;

[0017] Randomly select N based on the distribution characteristics of the input variables train Sample points x i , calculate N train The output response corresponding to the sample is given noise, that is, data uncertainty. The output response containing noise is as follows:

[0018]

[0019] Where ε represents the data noise caused by measurement accuracy, experimental error, etc., i = 1, 2, ..., N train .

[0020] Furthermore, the data-driven neural network model is a fully connected neural network model or a convolutional neural network;

[0021] When the data-driven neural network model is a fully connected neural network model, the input of the data-driven neural network model is x i , the output of the data-driven neural network model is where i=1,2,…,N train ;

[0022] The loss function of the data-driven neural network model is the mean square error function, and the loss Loss of the data-driven neural network is:

[0023]

[0024] When the loss of the data-driven neural network no longer decreases, the neural network training is completed.

[0025] Furthermore, the neural network model embedded with physical information is a fully connected neural network model or a convolutional neural network.

[0026] Furthermore, the hidden layer, number of neurons, activation function, learning rate, and optimizer parameters of the neural network model embedded with physical information are the same as the settings of the data-driven neural network model.

[0027] The input of the neural network model embedded with physical information is x i , the output of the neural network model embedded with physical information is

[0028] Furthermore, the loss function of the neural network model embedded with physical information is a mean square error function, and the total loss Loss of the neural network embedded with physical information is:

[0029]

[0030] Where Loss1 represents the physical loss from the input-output functional relationship based on the differential equation, Loss2 represents the data loss from the initial conditions and boundary conditions, and N obs Represents the number of observed samples.

[0031] Furthermore, the steps of obtaining the neural network model embedded with physical information improved based on the Bayesian method, the neural network model embedded with physical information improved based on the deep integration method, and the neural network model embedded with physical information improved based on MC Dropout are:

[0032] S4.1: Improved neural network model embedded with physical information based on Bayesian method;

[0033] A neural network model embedded with physical information improved based on the Bayesian method is based on a neural network model embedded with physical information; the parameters of the neural network model embedded with physical information improved based on the Bayesian method are random variables; the random variables include normal variables;

[0034] The training process of the neural network model embedded with physical information improved based on the Bayesian method is to optimize the distribution parameters of the parameters of the neural network model embedded with physical information improved based on the Bayesian method;

[0035] The parameters of the neural network model embedded with physical information improved based on the Bayesian method include the weight and bias of the network;

[0036] The distribution parameters include mean and variance;

[0037] S4.2: Improved neural network model embedded with physical information based on deep integration method;

[0038] Establishing N neural network models with the same physical information embedded in the architecture, and using different network parameter initializations to train the same training samples, thereby obtaining N neural network models with the physical information embedded. The N neural network models with the physical information embedded are the improved neural network models with the physical information embedded based on the deep integration method.

[0039] The same architecture settings include the same network structure and parameters, the network parameters include weights and biases, and the initialization adopts a random initialization method to ensure that the network trained N times has different initial parameters;

[0040] S4.3: Neural network model embedded with physical information based on MC Dropout improvement;

[0041] Dropout is turned on during the training phase of the neural network model embedded with physical information, so that the parameters of the neural network model embedded with physical information participate in the training with a probability value p each time, 0≤p≤1; the neural network model embedded with physical information after training N times is the neural network model embedded with physical information improved by the MC Dropout method.

[0042] Furthermore, the selection strategy is:

[0043] The selection principle based on the distribution range of training samples is:

[0044] Within the distribution range of training samples, the uncertainties of the neural network model with embedded physical information improved by the deep integration method and the neural network model with embedded physical information improved by MC Dropout are both smaller than that of the neural network model with embedded physical information improved by the Bayesian method.

[0045] Outside the distribution range of training samples, the uncertainty of the neural network model embedded with physical information improved by Bayesian method is smaller than that of the neural network model embedded with physical information improved by deep integration method and the neural network model embedded with physical information improved by MC Dropout.

[0046] The selection principle based on deviation is:

[0047] The deviation measures the prediction accuracy. The prediction accuracy of the neural network model embedded with physical information improved by the Bayesian method is higher than that of the neural network model embedded with physical information improved by the deep integration method and the neural network model embedded with physical information improved by MC Dropout.

[0048] The selection principle based on variance is:

[0049] The variance measures the prediction robustness; the prediction robustness is ranked from high to low as follows: the neural network model with embedded physical information improved by the deep integration method, the neural network model with embedded physical information improved by MC Dropout, and the neural network model with embedded physical information improved by the Bayesian method;

[0050] The principles for selecting training time are:

[0051] The training time measures the computational cost; according to the training time, the computational cost is ranked from low to high as the neural network model embedded with physical information improved based on MCDropout, the neural network model embedded with physical information improved based on the Bayesian method, and the neural network model embedded with physical information improved based on the deep integration method.

[0052] The beneficial effects of the present invention are: in view of the problem that data uncertainty cannot be eliminated, by constructing a neural network embedded with physical information, it is possible to accurately reflect actual physical phenomena, reduce dependence on data, thereby reducing errors caused by data uncertainty and improving the reliability of predictions. By comparing with the traditional data-driven neural network model, it can be seen that the fitting effect of the present invention is better and is less affected by data uncertainty. In view of the problem that model uncertainty methods are difficult to select and promote, the present invention provides a systematic comparison and selection scheme for model uncertainty quantification methods, and provides a comparison of the effects of three model uncertainty quantification methods from four perspectives: training sample distribution range, deviation, variance, and training time. Furthermore, the selection criteria for uncertainty quantification methods are clarified, which can more efficiently evaluate and optimize models, save time and resources, and improve overall research efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a simplified flow chart of the present invention;

[0054] Figure 2 Schematic diagram of the structure of a single-degree-of-freedom undamped vibration system in an embodiment of the present invention; (a) is a spring oscillator system; (b) is a shaft-disk torsion system; (c) is a simple pendulum system;

[0055] Figure 3 is a schematic diagram of the bounded heat transfer rod structure in an embodiment of the present invention;

[0056] Figure 4 A comparison diagram of prediction results of a data-driven neural network model and a neural network model embedded with physical information in the first embodiment of the present invention;

[0057] Figure 5 The first embodiment of the present invention is a comparison diagram of the prediction results of the three improved neural network models embedded with physical information, namely, the Bayesian method, the deep integration method and the MC Dropout method; (a) is the prediction result of the neural network model embedded with physical information improved by the Bayesian method; (b) is the prediction result of the neural network model embedded with physical information improved by the deep integration method; (c) is the prediction result of the neural network model embedded with physical information improved by the MC Dropout method;

[0058] Figure 6The deviation and variance of the prediction results of the three improved neural network models embedded with physical information, namely, the Bayesian method, the deep integration method and the MC Dropout, in the first embodiment of the present invention; (a) is a deviation graph; (b) is a variance graph;

[0059] Figure 7 1 is a comparison diagram of the prediction results of the data-driven neural network model and the neural network model embedded with physical information in the second embodiment of the present invention; (a) is a comparison diagram of the prediction results of the data-driven neural network model; (b) is a comparison diagram of the prediction results of the neural network model embedded with physical information; (c) is a comparison diagram of the exact solution;

[0060] Figure 8 The following are comparison diagrams of prediction results of three improved neural network models embedded with physical information, namely, the Bayesian method, the deep integration method and the MC Dropout method, in the second embodiment of the present invention; (a) is a comparison diagram of the Bayesian method results; (b) is a comparison diagram of the deep integration method; (c) is a comparison diagram of the MC Dropout method;

[0061] Fig. 9 In the second embodiment of the present invention, the deviation and variance of the prediction results of the three improved neural network models embedded with physical information, namely the Bayesian method, the deep integration method and the MC Dropout method, are shown; (a) is a deviation comparison chart; (b) is a variance comparison chart. DETAILED DESCRIPTION

[0062] An uncertainty quantification method based on a neural network embedded with physical information includes a neural network model embedded with physical information and a model uncertainty quantification selection method based on improving the neural network model embedded with physical information; the steps of constructing the neural network model embedded with physical information are:

[0063] S1: For the model or system to be evaluated, the input of the model or system to be evaluated is x, the output of the model or system to be evaluated is y, and the functional relationship between input and output is y=u(x), which is governed by the following differential equation:

[0064]

[0065] In the formula, represents the differential operator, and f(x) is a function of known form.

[0066] Randomly select N based on the distribution characteristics of the input variables train Sample points x i , calculate N train The output response corresponding to the sample is given noise, that is, data uncertainty. The output response containing noise is as follows:

[0067]

[0068] Where ε represents the data noise caused by measurement accuracy, experimental error, etc., i = 1, 2, ..., N train ;

[0069] S2: Based on N in step S1 train Input-output samples with data noise are used to build and train a data-driven neural network model;

[0070] The data-driven neural network model is a fully connected neural network model or a convolutional neural network;

[0071] The input of the data-driven neural network model is x i , the output of the data-driven neural network model is where i=1,2,…,N train ;

[0072] The loss function of the data-driven neural network model is the mean square error function, and the loss Loss of the data-driven neural network is:

[0073]

[0074] When the loss of the data-driven neural network no longer decreases, the neural network training is completed;

[0075] S3: Based on the physical equations, initial conditions and boundary conditions satisfied by the input-output variables, a neural network model embedded with physical information is established and trained;

[0076] The neural network model embedded with physical information is a fully connected neural network model or a convolutional neural network. The input of the neural network model embedded with physical information is x i , the output of the neural network model embedded with physical information is i=1,2,…,N train The hidden layer, number of neurons, activation function, learning rate, and optimizer parameters of the neural network model embedded with physical information are the same as those of the data-driven neural network model.

[0077] The loss function of the neural network model embedded with physical information is the mean square error function, and the total loss Loss of the neural network embedded with physical information is:

[0078]

[0079] Where Loss1 represents the physical loss from the differential equation, Loss2 represents the data loss from the initial and boundary conditions, and N obs represents the number of observed samples;

[0080] The model uncertainty quantification selection method based on improving the neural network model embedded with physical information includes the following steps:

[0081] S4: Based on the neural network model embedded with physical information established in step S3, the Bayesian method, the deep integration method and the MC Dropout method are used to improve it respectively;

[0082] S4.1: Improvement of neural network models embedding physical information using Bayesian methods;

[0083] The parameters of the neural network model embedded with physical information are modified from determined values ​​to random variables, and the training process of the neural network model embedded with physical information is changed from optimizing the parameters of the neural network model embedded with physical information to optimizing the distribution parameters of the parameters of the neural network model embedded with physical information;

[0084] The neural network model parameters embedded with physical information include network weights and biases;

[0085] The random variable comprises a normal variable;

[0086] The distribution parameters include mean and variance;

[0087] S4.2: Improve neural network models that embed physical information using deep ensemble methods;

[0088] Establish N neural network models with the same architecture settings and embed physical information, and use different network parameter initializations to train the same training samples to obtain N integrated neural network models with embedded physical information;

[0089] The same architecture settings include the same network structure and parameters, the network parameters include weights and biases, and the initialization adopts a random initialization method to ensure that the network trained N times has different initial parameters;

[0090] S4.3: Use MC Dropout method to improve the neural network model embedded with physical information;

[0091] In the training stage of the neural network model embedded with physical information, Dropout is enabled, so that the parameters of the neural network model embedded with physical information participate in the training with probability value p each time, 0≤p≤1. The neural network model embedded with physical information after training N times is the neural network model embedded with physical information improved by MC Dropout method.

[0092] S5: Based on the improved neural network model embedded with physical information, the same data set x i ,i=1,2,…,N test Make predictions;

[0093] S5.1: Based on the improved neural network model embedded with physical information based on the Bayesian method, for the same test data x i ,i=1,2,…,N test Perform N predictions to obtain the prediction results of the neural network model embedded with physical information based on the Bayesian method. The prediction results include mean, variance, deviation and training time.

[0094] S5.2: Based on the improved neural network model embedded with physical information based on the deep integration method, for the same test data x i ,i=1,2,…,N test Make predictions and obtain the prediction results of the neural network model embedded with physical information based on the deep integration method. The prediction results include mean, variance, deviation and training time.

[0095] S5.3: Based on the improved neural network model embedded with physical information after MC Dropout, for the same test data x i ,i=1,2,…,N test Perform N predictions to obtain the prediction results of the neural network model embedded with physical information based on MC Dropout. The prediction results include mean, variance, deviation and training time.

[0096] S6: Based on the prediction results of the three improved methods in step S5, the improved embedded physical information neural network model is selected according to the training sample range requirements, deviation requirements, variance requirements and training time requirements of the engineering requirements;

[0097] Selection principles:

[0098] Within the distribution range of training samples, the uncertainty of the neural network model embedded with physical information based on deep integration and MC Dropout is smaller than that of the Bayesian neural network model embedded with physical information. Outside the distribution range of training samples, the uncertainty of the Bayesian neural network model embedded with physical information is smaller than that of the neural network model embedded with physical information based on deep integration and MC Dropout.

[0099] The deviation measures the prediction accuracy. The prediction accuracy of the Bayesian neural network model embedded with physical information is higher than that of the neural network model embedded with physical information improved by deep integration and MC Dropout.

[0100] The variance measures the prediction robustness. The prediction robustness of the three methods is ranked from high to low as deep integration method, MC Dropout method, and Bayesian method.

[0101] The training time measures the computational cost. From the training time, we can see that the computational costs of the three methods are ranked from low to high as MC Dropout method, Bayesian method, and deep integration method.

[0102] Two exemplary illustrations are given below.

[0103] The first one is a single-degree-of-freedom undamped vibration system, refer to Figure 2 As shown. Poisson's equation is an ordinary differential equation in mathematics that is commonly found in electrostatics, mechanical engineering and theoretical physics. Therefore, it is widely used in the engineering field. The spring oscillator system is a typical single-degree-of-freedom undamped vibration system, and its motion process can be described by Poisson's equation. Based on the neural network, the displacement of the spring oscillator at different times is predicted, and the data uncertainty and model uncertainty are quantified, which can be extended to engineering problems in other mechanical fields. The second one is the heat conduction problem. The object of study is a bounded heat transfer rod. Figure 3 As shown. The temperature distribution inside the rod can be described by the diffusion equation. Based on the neural network, the temperature of the bounded rod at different times and positions is predicted, and the data uncertainty and model uncertainty are quantified, which can be extended to other heat conduction and flow field problems.

[0104] This example implementation aims to verify the uncertainty quantification method based on the classic problem, provide guidance for the selection and application of the model uncertainty quantification method, and reduce the uncertainty of the data. Figure 2 As shown, the input-output function of the spring oscillator is recorded as:

[0105]

[0106] Where ε is the data noise that describes the output uncertainty.

[0107] The Poisson equation satisfied by the input-output variables is recorded as:

[0108]

[0109] refer to Figure 3 As shown, the temperature distribution function in the bounded rod is recorded as:

[0110] u=sin(2πx)e -t +ε (11)

[0111] The diffusion equation satisfied by the input-output variables is recorded as:

[0112]

[0113] In step S1, N are randomly selected based on the distribution characteristics of the input variables. train Sample points, calculate this N trainThe output response quantity corresponding to each sample is given noise, i.e., data uncertainty. In the first embodiment, the input variable is time t, which can correspond to a set of sample points uniformly selected within the time interval. The output response quantity is the displacement u of the spring oscillator, which can be calculated by formula (1). The noise is realized by the variable ε. In the second embodiment, the input variable is the distance x between the rod and the left end at time t, which can correspond to a set of sample points uniformly selected within the time interval and length space. The output response quantity is the temperature u inside the rod, which can be calculated by formula (3). The noise is realized by the variable ε.

[0114] In step S2, based on N in step S1 train Input-output samples containing data noise are used to establish and train a data-driven neural network model. In this example implementation, the neural network model includes a fully connected neural network based on N train A neural network model is trained using a set of noisy input-output samples.

[0115] In step S3, based on the physical equations satisfied by the input-output variables and the samples in step S1, a neural network model embedded with physical information is established and trained. In this example implementation, the physical equations satisfied by the input-output variables include Poisson's equation (2) and heat conduction diffusion equation (4).

[0116] For the first embodiment, the output of the sample in step S1 is calculated based on the neural network. Use automatic differentiation technology to calculate the partial derivative corresponding to the sample output in step S1

[0117] Construct the loss based on the initial conditions:

[0118]

[0119] The loss is constructed based on Poisson equation (2):

[0120]

[0121] For the second embodiment, the loss functions shown in equations (5) and (6) are expressed as:

[0122]

[0123]

[0124] Based on the total loss Loss = Loss data +Loss phy Train the neural network to obtain a neural network model that embeds physical information.

[0125] Based on the data-driven neural network model in step S2 and the neural network model embedded with physical information in step S3, the randomly selected N test The prediction results of the two neural network models are compared. Figure 4 and Figure 7 It can be seen that the prediction uncertainty of the neural network model embedded with physical information is smaller. This is because the neural network model embedded with physical information no longer requires input-output samples, which reduces the dependence on data and thus reduces the impact of data uncertainty on the prediction results.

[0126] In step S4, based on the neural network model embedded with physical information established in step S3, it is improved using the Bayesian method, the deep integration method and the MC Dropout method respectively.

[0127] In this example implementation, the main process of using the Bayesian method to improve the neural network model embedded with physical information is to change the neural network parameters from fixed values ​​to random variables. Based on this, the form of the loss function is:

[0128] Loss PINN =E q [logq(W)-logp(W)]-E q [logp(Y|t,W)] (17)

[0129] In the formula, Loss PINN is the loss function of the neural network model embedded with physical information; W is the neural network parameter, including weights and biases; p(W) is the prior probability of parameter W; q(W) is the probability of parameter W under the assumption of normal distribution, including but not limited to; E q [·] is the expectation operator about q; p(Y|t,W) is the probability of output under given network parameters and input, which includes two parts: initial conditions and differential equations:

[0130] The neural network is trained based on the loss function shown in formula (9) to obtain a Bayesian neural network model embedded with physical information.

[0131] In this example implementation, the main process of improving the neural network model embedded with physical information using the deep integration method is to establish N neural network models with the same settings of embedded physical information, and use different network parameter initializations for training on the same training samples to obtain an integrated neural network model embedded with physical information.

[0132] In this example implementation, the main process of using the MC Dropout method to improve the neural network model embedded with physical information is to enable Dropout during the neural network training phase so that the neural network parameters participate in the training each time with a certain probability value p (0≤p≤1).

[0133] In step S5, based on the improved neural network model embedded with physical information, the same data is predicted, and the variance of the prediction results and the deviation between the mean of the prediction results and the true value are compared.

[0134] In this example implementation, for the Bayesian neural network model embedded with physical information, N random samples are drawn according to the distribution of network parameters to obtain N different prediction results; for the integrated neural network model embedded with physical information, N integrated neural networks are used for prediction to obtain N different prediction results; for the integrated neural network model embedded with physical information improved by the MC Dropout method, Dropout is turned on in the test phase, and N predictions are made to obtain N different prediction results. The prediction results obtained based on the three improved methods are shown in Figure 2. Figure 5 and Figure 8 As shown. Solve for its mean, variance and deviation from the true value, the result is as follows Figure 6 and Fig. 9 shown.

[0135] In step S6, based on the bias and variance of the prediction results of the three improved methods in step S5, guidance is provided for the selection of a method for quantifying model uncertainty.

[0136] In this exemplary embodiment, in order to demonstrate the beneficial effects of the technical solution of the present invention, Figure 5 and Figure 6 A set of comparisons of model uncertainty quantification methods for spring oscillator systems are given. Figure 8 and Fig. 9 A set of model uncertainty quantification methods for heat transfer rods are compared. It can be seen that within the distribution range of training samples, the uncertainty of the neural network model embedded with physical information based on deep integration and MC Dropout is similar, and both are smaller than the Bayesian neural network model embedded with physical information; outside the distribution range of training samples, the uncertainty of the Bayesian neural network model embedded with physical information is smaller than the other two.

[0137] The deviation measures the prediction accuracy. From the deviation, we can see that the prediction accuracy of the Bayesian neural network model embedded with physical information is the highest, and the prediction accuracy of the neural network model embedded with physical information based on deep integration and MC Dropout improvement is similar.

[0138] The variance measures the prediction robustness. From the variance, we can see that the prediction robustness of the three methods is ranked from high to low as deep integration method, MC Dropout method, and Bayesian method.

[0139] The training time measures the computational cost. From the training time, we can see that the computational costs of the three methods are ranked from low to high as MC Dropout method, Bayesian method, and deep integration method.

[0140] Finally, it should be noted that the present invention is not limited to the precise structures and problems described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from the scope thereof. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. An uncertainty quantification method based on a neural network embedded with physical information, characterized by: The invention comprises a neural network model embedded with physical information and a model uncertainty quantification selection method based on improving the neural network model embedded with physical information; the steps of constructing the neural network model embedded with physical information are as follows: S1: Establish the input-output functional relationship of the model or system to be evaluated based on the differential equation, and calculate the noise-containing response output of the input-output functional relationship based on the differential equation according to the random distribution characteristics of the input variables of the model or system to be evaluated; S2: Based on the input-output functional relationship of the differential equation, the noisy input-output samples are established to build and train the data-driven neural network model; S3: Establish and train a neural network model embedded with physical information based on the input-output functional relationship, initial conditions and boundary conditions of the model or system to be evaluated by differential equations; The steps of the model uncertainty quantization selection method based on the neural network model embedded with physical information are as follows: S4: Based on the established neural network model embedded with physical information, the Bayesian method, the deep integration method and the MC Dropout method are used to improve it, and the neural network model embedded with physical information improved by the Bayesian method, the neural network model embedded with physical information improved by the deep integration method and the neural network model embedded with physical information improved by the MC Dropout are obtained respectively; S5: For the neural network model embedded with physical information improved by the Bayesian method, the neural network model embedded with physical information improved by the deep integration method, and the neural network model embedded with physical information improved by MC Dropout, the same data set is used for prediction, and the prediction results of the neural network model embedded with physical information improved by the Bayesian method, the prediction results of the neural network model embedded with physical information improved by the deep integration method, and the prediction results of the neural network model embedded with physical information improved by MC Dropout are obtained respectively; The prediction results of the neural network model embedded with physical information improved based on the Bayesian method, the prediction results of the neural network model embedded with physical information improved based on the deep integration method, and the prediction results of the neural network model embedded with physical information improved based on MC Dropout all include mean, variance, deviation and training time; S6: Based on the prediction results of the neural network model embedded with physical information improved by the Bayesian method, the prediction results of the neural network model embedded with physical information improved by the deep integration method, and the prediction results of the neural network model embedded with physical information improved by MC Dropout, according to engineering requirements and selection strategies, the improved neural network model embedded with physical information is selected as the optimal neural network model embedded with physical information for model uncertainty quantification.

2. The uncertainty quantification method based on a neural network embedded with physical information according to claim 1 is characterized in that: The specific steps for calculating the noisy response output based on the input-output functional relationship of the differential equation are: For the model or system to be evaluated, the input of the model or system to be evaluated is x, the output of the model or system to be evaluated is y, and the functional relationship between input and output is y=u(x), which is governed by the following differential equation: l x u(x)=f(x) (1) In the formula, l x represents the differential operator, f(x) is a function of known form; Randomly select N based on the distribution characteristics of the input variables train Sample points x i , calculate N train The output response corresponding to the sample is given noise, that is, data uncertainty. The output response containing noise is as follows: Where ε represents the data noise caused by measurement accuracy, experimental error, etc., i = 1, 2, ..., N train .

3. The uncertainty quantification method based on a neural network embedded with physical information according to claim 1 is characterized in that: The data-driven neural network model is a fully connected neural network model or a convolutional neural network.

4. The uncertainty quantification method based on a neural network embedded with physical information according to claim 1, characterized in that: When the data-driven neural network model is a fully connected neural network model, the input of the data-driven neural network model is x i , the output of the data-driven neural network model is where i=1,2,…,N train ; The loss function of the data-driven neural network model is the mean square error function, and the loss Loss of the data-driven neural network is: When the loss of the data-driven neural network no longer decreases, the neural network training is completed.

5. The uncertainty quantification method based on a neural network embedded with physical information according to claim 1, characterized in that: The neural network model embedded with physical information is a fully connected neural network model or a convolutional neural network; the hidden layer, number of neurons, activation function, learning rate, and optimizer parameters of the neural network model embedded with physical information are the same as the settings of the data-driven neural network model.

6. The uncertainty quantification method based on a neural network embedded with physical information according to claim 1, characterized in that: The input of the neural network model embedded with physical information is x i , the output of the neural network model embedded with physical information is i=1,2,…,N train ; The loss function of the neural network model embedded with physical information is the mean square error function, and the total loss Loss of the neural network embedded with physical information is: Where Loss1 represents the physical loss from the input-output functional relationship based on the differential equation, Loss2 represents the data loss from the initial conditions and boundary conditions, and N obs Represents the number of observed samples.

7. The uncertainty quantification method based on a neural network embedded with physical information according to claim 1, characterized in that: The steps of obtaining the neural network model embedded with physical information improved based on the Bayesian method, the neural network model embedded with physical information improved based on the deep integration method, and the neural network model embedded with physical information improved based on MC Dropout are: S4.1: Improved neural network model embedded with physical information based on Bayesian method; A neural network model embedded with physical information improved based on the Bayesian method is based on a neural network model embedded with physical information; the parameters of the neural network model embedded with physical information improved based on the Bayesian method are random variables; the random variables include normal variables; The training process of the neural network model embedded with physical information improved based on the Bayesian method is to optimize the distribution parameters of the parameters of the neural network model embedded with physical information improved based on the Bayesian method; The parameters of the neural network model embedded with physical information improved based on the Bayesian method include the weight and bias of the network; the distribution parameters include the mean and variance; S4.2: Improved neural network model embedded with physical information based on deep integration method; Establishing N neural network models with the same physical information embedded in the architecture, and using different network parameter initializations to train the same training samples, thereby obtaining N neural network models with the physical information embedded. The N neural network models with the physical information embedded are the improved neural network models with the physical information embedded based on the deep integration method. The same architecture settings include the same network structure and parameters, the network parameters include weights and biases, and the initialization adopts a random initialization method to ensure that the network trained N times has different initial parameters; S4.3: Neural network model embedded with physical information based on MC Dropout improvement; Dropout is turned on during the training phase of the neural network model embedded with physical information, so that the parameters of the neural network model embedded with physical information participate in the training with a probability value p each time, 0≤p≤1; the neural network model embedded with physical information after training N times is the neural network model embedded with physical information improved by the MC Dropout method.

8. The uncertainty quantification method based on a neural network embedded with physical information according to claim 1, characterized in that: The selection strategies include selection principles based on the distribution range of training samples, selection principles based on deviations, selection principles based on variances, and selection principles based on training time; The selection principle based on the distribution range of training samples is: Within the distribution range of training samples, the uncertainties of the neural network model with embedded physical information improved by the deep integration method and the neural network model with embedded physical information improved by MC Dropout are both smaller than that of the neural network model with embedded physical information improved by the Bayesian method. Outside the distribution range of training samples, the uncertainty of the neural network model embedded with physical information improved by Bayesian method is smaller than that of the neural network model embedded with physical information improved by deep integration method and the neural network model embedded with physical information improved by MC Dropout. The selection principle based on deviation is: The deviation measures the prediction accuracy. The prediction accuracy of the neural network model embedded with physical information improved by the Bayesian method is higher than that of the neural network model embedded with physical information improved by the deep integration method and the neural network model embedded with physical information improved by MCDropout. The selection principle based on variance is: The variance measures the prediction robustness; the prediction robustness is ranked from high to low as follows: the neural network model with embedded physical information improved by the deep integration method, the neural network model with embedded physical information improved by MC Dropout, and the neural network model with embedded physical information improved by the Bayesian method; The principles for selecting training time are: The training time measures the computational cost; according to the training time, the computational cost is ranked from low to high as the neural network model embedded with physical information improved based on MCDropout, the neural network model embedded with physical information improved based on the Bayesian method, and the neural network model embedded with physical information improved based on the deep integration method.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, an uncertainty quantification method based on a neural network embedded with physical information as described in any one of claims 1 to 8 is implemented.

Images