Energy constraint time prediction and uncertainty quantification method based on Bayesian neural network

Abstract

The invention discloses an energy constraint time prediction and uncertainty quantification method based on a Bayesian neural network, and relates to the field of machine learning and nuclear fusion physics. The method comprises the following steps: firstly, constructing a data set according to experimental data of a Tokamak device, determining physical variables and device parameters for predicting, and preprocessing the data; secondly, constructing a Bayesian neural network model based on variational reasoning, and adjusting model parameters through training; and finally, using the trained variational reasoning Bayesian neural network model to output an energy constraint time prediction value and an uncertainty quantification result. The invention aims to solve the problem that the traditional method is low in tokamak energy constraint time prediction precision and cannot quantify uncertainty.

Description

Technical Field

[0001] The present invention relates to the fields of machine learning and fusion physics. Specifically, it relates to a method for predicting energy confinement time and quantifying uncertainty based on a Bayesian neural network. Background Art

[0002] Tokamak devices are one of the important devices for realizing controlled nuclear fusion reactions. The tokamak energy confinement time is an important parameter for measuring the ability of a tokamak device to maintain the energy of the plasma, indicating the length of time that the plasma can maintain its energy in the tokamak device. It is usually used to evaluate the confinement performance of the plasma. The longer the energy confinement time, the slower the energy loss of the plasma and the higher the fusion efficiency of the device. Traditional methods for predicting energy confinement time are mainly based on empirical formulas, establishing the relationship between the total energy confinement time of the plasma and important parameters such as magnetic field strength, plasma density, ion species, total current, effective heating power, and device size through statistical methods to predict the energy confinement time, that is, the empirical scaling law. This method is easy to apply, but it cannot handle the complex non-linear relationships between these physical parameters, resulting in poor prediction accuracy.

[0003] In recent years, machine learning methods, especially neural networks, have been widely used in the research of tokamak plasma physics and have shown good prediction performance. However, traditional empirical formulas and neural network models are usually deterministic models that can only provide point estimation results and cannot quantify the uncertainty of the prediction results, which limits their application in the operation optimization and decision support of tokamak devices.

[0004] There are almost no patent applications for predicting energy confinement time and quantifying uncertainty in the prior art. This research proposes a new data-driven model based on a Bayesian neural network, innovating against the disadvantages of poor prediction accuracy and inability to quantify uncertainty of traditional methods and deterministic models. Summary of the Invention

[0005] In view of the above defects of the existing methods, the technical problem to be solved by the present invention is how to provide more accurate prediction results and quantify uncertainty to achieve more accurate and reliable prediction of tokamak energy confinement time.

[0006] To achieve the above object, the technical solution of the present invention is: A method for predicting energy confinement time and quantifying uncertainty based on a Bayesian neural network, the method comprising the following steps:

[0007] Step 1: Dataset construction and preprocessing, including: obtaining relevant experimental data of the tokamak device, selecting diagnostic signals and parameters as inputs to predict the energy confinement time, and preprocessing the data, including feature screening, data standardization, and dataset partitioning, to obtain the basic datasets for network training, validation, and testing, including the training set, validation set, and test set;

[0008] Step 2: Construct a Bayesian neural network model, including: constructing a Bayesian neural network model, selecting the weight parameters of the Bayesian neural network as the prior distribution, and parameterizing the weight parameters of the Bayesian neural network through variational parameters to approximate the variational posterior distribution of the weight parameters of the Bayesian neural network. The Bayesian neural network model is a fully connected network structure including an input layer, a hidden layer, and an output layer. The input layer receives the experimental data of the tokamak device, and the output layer outputs the predicted value distribution of the energy confinement time;

[0009] Step 3: Conduct model training, including: calculating the KL divergence between the prior distribution and the variational posterior distribution of the weights of the Bayesian neural network, and calculating the expected log-likelihood according to the training dataset, constructing an objective function, and training the variational inference parameters by maximizing the objective function;

[0010] Step 4: Use the trained Bayesian neural network model to predict the results and quantify the uncertainty.

[0011] The present invention has the following beneficial effects:

[0012] Compared with the existing empirical scaling law method, the model of the present invention has greatly improved the accuracy of predicting the energy confinement time and can describe the uncertainty of predicting the energy confinement time, thus providing reliable information for optimizing fusion experiments. Brief Description of the Drawings

[0013] The following further describes the present invention in combination with the drawings and embodiments.

[0014] Figure 1 is the flow chart of the embodiment of the present invention;

[0015] Figure 2 is the scatter plot predicted by the model of the embodiment of the present invention on the STD5 dataset;

[0016] Figure 3 is the uncertainty analysis diagram of predicting the energy confinement time on the test sets of the JET, ASDEX-Upgrade, and DIII-D devices provided by the embodiment of the present invention. Detailed Embodiments

[0017] To more clearly elaborate the object, technical solution and effective gain of the present invention, the following further describes the present invention in combination with specific implementation cases and attached drawings. In addition, the specific implementation examples described below are only used to explain the present invention, but are not limited thereto. Further, the technical features involved in the various embodiments of the present invention described below can be combined with each other without conflict.

[0018] As Figure 1 shown, a method for energy confinement time prediction and uncertainty quantification based on a Bayesian neural network, the method comprising the following steps:

[0019] Step 1, dataset construction and preprocessing: Obtain relevant experimental data of a tokamak device, select parameters such as plasma current, average electron density, and elongation of the closed magnetic surface as input variables, select the energy confinement time as the output variable, and preprocess the data, including feature screening, data normalization, and set partitioning, to obtain a basic dataset for network training, verification, and testing, including a training set, a verification set, and a test set;

[0020] Step 2, construct a Bayesian neural network model: Construct a Bayesian neural network model, select the weight parameters of the Bayesian neural network as the prior distribution, parameterize the weight parameters of the Bayesian neural network through variational parameters to approximate the posterior distribution of the weight parameters of the Bayesian neural network. The Bayesian neural network model is a fully connected network structure including an input layer, a hidden layer, and an output layer. The input layer receives the experimental data of the tokamak device, and the output layer outputs the predicted value distribution of the energy confinement time;

[0021] Step 3, perform model training: Calculate the KL divergence between the prior distribution of the weights of the Bayesian neural network and the variational posterior distribution, and calculate the expected log-likelihood according to the training dataset, construct an objective function, and train the variational inference parameters by maximizing the objective function;

[0022] Step 4, use the trained Bayesian neural network model to predict the results and quantify the uncertainty.

[0023] Specifically, in Step 1, data from the ITPA global H-mode confinement database (DB5.2.3) was used to construct a dataset STD5 containing multiple tokamak devices. STD5 includes 3 single-device datasets, and the 3 tokamak devices are JET, ASDEX-Upgrade, and DIII-D. Nine key diagnostic signals and parameters in the data were used as inputs to predict the energy confinement time. These signals include plasma current, toroidal magnetic field in vacuum, major radius of the plasma, average electron density, thermal power loss, elongation of the LCFS, inverse aspect ratio, and effective atomic mass M. effAnd the average three angles of LCFS, and the output is the energy confinement time. Preprocess all data sets, remove unreasonable and missing-value data, and divide the data sets. Among them, 70% of the data is allocated to the training set, 10% to the test set, and 20% to the validation set.

[0024] Specifically, in step 2, the network structure of the Bayesian neural network model sets two hidden layers, and the number of nodes in each layer is 50. The prior probability distribution of the weights w of the Bayesian neural network is selected as the standard normal distribution , where is the weight parameter of the i-th layer and j-th dimension of the Bayesian neural network.

[0025] Specifically, in step 3, the variational posterior distribution adopts a Gaussian distribution with as the mean and as the standard deviation. Its distribution can be expressed as follows:

[0026] ;

[0027] where is the mean of the weight parameter of the i-th layer and j-th dimension of the Bayesian neural network , and is the variance of the weight parameter of the i-th layer and j-th dimension of the Bayesian neural network .

[0028] Specifically, the calculation formula of the KL divergence between the prior distribution and the variational posterior distribution of the weight parameters of the Bayesian neural network is as follows:

[0029] ;

[0030] where is the mean of the weight parameter of the i-th layer and j-th dimension of the Bayesian neural network , and is the variance of the weight parameter of the i-th layer and j-th dimension of the Bayesian neural network .

[0031] Specifically, the expected log-likelihood is approximated according to the following formula:

[0032] ;

[0033] ;

[0034] where represents the set of input experimental variables , represents the set of energy confinement times , ( ) represents the th different data point randomly selected from all data sets, = 1, 2, … M, where M is the total number of randomly selected data points and N is the number of data points in the training set, represents the weight parameter of the jth dimension of the ith layer network obtained by sampling, is the weight of the model, is the weight parameter of the jth dimension of the ith layer of the Bayesian neural network mean, represents the variational parameter, is a random number that follows a standard normal distribution , represents the parametric function.

[0035] The specific objective function is the evidence lower bound , and the model parameters are optimized by maximizing , The formula of

[0036] ;

[0037] where, is the expected log-likelihood of the data under the variational distribution, which is used to measure the fitting degree of the model to the data, is the KL divergence between the variational distribution and the prior distribution, and minimizing this term makes the variational posterior distribution closer to the true posterior distribution.

[0038] Specifically, the model training uses the forward propagation algorithm to calculate the value of the objective function , and then uses the backward propagation algorithm to calculate the partial derivative of the variational parameter . The variational parameter is optimized by the Adam optimizer, making the variational distribution as the approximate posterior distribution,

[0039] Specifically, in step 4, using the trained Bayesian neural network model to predict the energy constraint time and quantify the uncertainty includes the following steps:

[0040] Sample the weight parameter w of the model by the Monte Carlo sampling method, set the number of samples to 50, and obtain the prediction result set of the energy constraint time through the model calculation.

[0041] Calculate the mean and standard deviation of the data in this set. The mean is the predicted result, and the standard deviation is used as the range of prediction uncertainty, and visualize the prediction and uncertainty effects.

[0042] Table 1 shows the evaluation index values of the Bayesian neural network model and the ITPA20-IL method for predicting the energy confinement time on the test set. Among them, RMSE represents the root mean square error, MAE represents the mean absolute error, and R 2 represents the coefficient of determination. It can be seen from Table 1 that the prediction results using VIBNN can obtain higher prediction accuracy.

[0043] Table 1 Evaluation index table for predicting energy confinement time on different data sets

[0044] Figure 2 Shows the scatter plot of the Bayesian neural network model VIBNN and the traditional scaling law model ITPA20-IL in the STD5 data set prediction. Figure 2 (a) is the scatter plot of the VIBNN prediction result, Figure 2 (b) is the scatter plot of the calculation result of the scaling law ITPA20-IL. It can be seen that the results predicted by VIBNN are more concentrated near the diagonal, and the predicted R 2 is 0.97, which is much larger than 0.87 of ITPA20-Il, indicating that VIBNN can better explain the nonlinear relationship between variables.

[0045] Figure 3 Shows the results of VIBNN quantifying uncertainty. The horizontal axis is the data index, showing the fitting curve of the true value and the predicted value and the 95% confidence interval. Figure 3 (a) is the result of VIBNN quantifying uncertainty in the JET device data set, Figure 3 (b) is the result of VIBNN quantifying uncertainty in the ASDEX-Upgrade device data set, Figure 3 (c) is the result of VIBNN quantifying uncertainty in the DIII-D device data set. It can be seen that there is a good correspondence between the true value, the predicted value and the confidence interval, and most of the data are distributed within the confidence interval, and a small number are near the confidence interval boundary.

Claims

1. An energy-constrained time prediction and uncertainty quantification method based on a Bayesian neural network, characterized in that The method includes the following steps: Step 1, dataset construction and preprocessing, including: obtaining relevant experimental data of the tokamak device, selecting diagnostic signals and parameters as inputs to predict the energy confinement time, and preprocessing the data, including feature screening, data normalization, and set partitioning, to obtain a basic dataset for network training, validation, and testing, including a training set, a validation set, and a test set; Step 2, constructing a Bayesian neural network model, including: constructing a Bayesian neural network model, selecting the prior distribution of the weight parameters of the Bayesian neural network, parameterizing the weight parameters of the Bayesian neural network through variational parameters to approximate the variational posterior distribution of the weight parameters of the Bayesian neural network. The Bayesian neural network model is a fully connected network structure including an input layer, a hidden layer, and an output layer. The input layer receives the experimental data of the tokamak device, and the output layer outputs the predicted value distribution of the energy confinement time; Step 3, model training, including: calculating the KL divergence between the prior distribution and the variational posterior distribution of the weights of the Bayesian neural network, and calculating the expected log-likelihood according to the training dataset, constructing an objective function, and training the variational inference parameters by maximizing the objective function; Step 4, using the trained Bayesian neural network model to predict results and quantify uncertainty.

2. A method for energy-constrained time prediction and uncertainty quantification based on a Bayesian neural network according to claim 1, characterized in that In Step 1, the diagnostic signals and parameters include plasma current, toroidal magnetic field in vacuum, major plasma radius, average electron density, thermal power loss, elongation of the Last Closed Flux Surface (LCFS), inverse aspect ratio, effective atomic mass M eff and the average three angles of the LCFS.

3. A method for energy-constrained time prediction and uncertainty quantification based on a Bayesian neural network according to claim 1, characterized in that In step 2, the network structure of the Bayesian neural network model is set with two hidden layers, and the prior probability distribution of the weights parameter w of the Bayesian neural network is selected as the standard normal distribution , where is the weights parameter of the i-th layer and the j-th dimension of the Bayesian neural network.

4. A method for energy-constrained time prediction and uncertainty quantification based on a Bayesian neural network according to claim 3, characterized in that, The number of nodes in each hidden layer is 50.

5. A method for energy-constrained time prediction and uncertainty quantification based on a Bayesian neural network according to claim 3, characterized in that In step 3, the variational posterior distribution is taken as the mean, and a Gaussian distribution with the standard deviation, and its distribution is expressed as follows: ； Among them, is the weight parameter of the j-th dimension of the i-th layer of the Bayesian neural network mean value of, is the j-th weight parameter of the i-th layer of the Bayesian neural network variance of.

6. A method for energy-constrained time prediction and uncertainty quantification based on a Bayesian neural network according to claim 5, characterized in that The formula for calculating the KL divergence between the prior distribution and the variational posterior distribution of the weight parameters of the Bayesian neural network is as follows: ； Among them, is the weight parameter of the j-th dimension of the i-th layer of the Bayesian neural network mean value of, is the j-th weight parameter of the i-th layer of the Bayesian neural network variance of.

7. A method for energy-constrained time prediction and uncertainty quantification based on a Bayesian neural network according to claim 6, characterized in that Expected log-likelihood Approximated according to the following formula: ； ； in, Represents the input experimental variable A collection of Energy constraint time The collection of ( ) means from all data sets Randomly selected from Different data points, =1, 2, ...M, M is the total number of randomly selected data points, N is the number of data in the training set, represents the weight parameter of the jth dimension of the i-th layer network obtained by sampling, represents the variational parameter, is a standard normal distribution A random number, Represents a parameterized function.

8. A method for energy-constrained time prediction and uncertainty quantification based on a Bayesian neural network according to claim 7, characterized in that, The objective function is the evidence lower bound , and the model parameters are optimized by maximizing . The formula for is as follows: ； wherein, is the expected log-likelihood of the data under the variational distribution, which is used to measure the fitting degree of the model to the data, is the KL divergence between the variational distribution and the prior distribution, and minimizing this term makes the variational posterior distribution closer to the true posterior distribution.

9. A method for energy-constrained time prediction and uncertainty quantification based on a Bayesian neural network according to claim 8, characterized in that The model training uses the forward propagation algorithm to calculate the value of the objective function and then uses the backward propagation algorithm to calculate the partial derivatives of the variational parameters . The variational parameters are optimized by the Adam optimizer, so that the variational distribution serves as an approximate posterior distribution . The learning rate for training is set to 0.0003, and the number of training epochs is 500.

10. A method for energy-constrained time prediction and uncertainty quantification based on a Bayesian neural network according to claim 8, characterized in that, In step 4, the weight parameters w of the model are sampled by the Monte Carlo sampling method, the number of samplings is set to 50, and a set of predicted results of the energy confinement time is calculated through the model.

Images