Slow-down neutron energy spectrum prediction method based on deep learning

Abstract

The invention discloses a moderated neutron energy spectrum prediction method based on deep learning, and the method comprises the steps: firstly obtaining a data set of neutron energy spectrum parameters through Monte Carlo simulation calculation, and carrying out the preprocessing of the data, including feature extraction, normalization, noise reduction and other means; then, a training set and a test set are constructed, a deep neural network model is constructed based on the multi-layer perceptron, and training is carried out through a small-batch training method; according to the model, a residual block is adopted to replace a hidden layer to enhance the learning ability of the model, and a dropout layer is utilized to prevent overfitting of the model. In the training process, model parameters are updated by calculating a mean square loss function and back propagation, so that the purposes of reducing prediction errors and improving prediction precision are achieved. The method can accurately predict the moderated neutron energy spectrum, and has an important value for application of artificial intelligence in the nuclear medicine field.

Description

Technical Field

[0001] The present invention belongs to the field of moderated neutron energy spectrum prediction, and more specifically relates to a moderated neutron energy spectrum prediction method based on deep learning. Background Art

[0002] The neutron energy spectrum is the distribution of the neutron fluence rate or the number of neutrons in a unit energy interval as the neutron energy changes. It contains a lot of information such as the neutron fluence rate and its energy, the total neutron fluence and the average energy. It can also obtain the neutron ambient dose equivalent and the absorbed dose based on the fluence-neutron dose equivalent conversion coefficient (h*(10)) or the absorbed dose coefficient. It is one of the most important parameters of the neutron radiation field.

[0003] Different neutron energy spectra are required for different application scenarios. The most common method is to slow down high-energy neutrons to a specific energy or energy range to form a neutron radiation field that meets the needs, such as reactor neutron energy spectrum, neutron reference radiation field for instrument detection and calibration, boron neutron capture therapy (Boron Neutron Capture Therapy, BNCT), etc. Calculating these neutron energy spectra requires the use of high-performance computers to perform a large number of Monte Carlo calculations, which is time-consuming and costly. If subsequent dose calculations are required based on the energy spectrum, such as in BNCT applications, the safety and accuracy of human irradiation doses must be ensured, the energy spectrum needs to be more accurate, which will undoubtedly further increase the time cost of the calculation.

[0004] Deep learning is a type of machine learning algorithm that imitates human neural networks. It uses deep neural networks to learn massive amounts of data and find the optimal solution to practical problems under multiple constraints. Multilayer perceptron (MLP) is a classic deep neural network model. It captures the complex interactions between inputs by adding multiple hidden layers to the network and applies a nonlinear activation function after the linear fully connected layer, so that the deep neural network model completes the transition from a linear model to a nonlinear model.

[0005] The specific neutron energy spectrum is usually obtained by slowing down fast neutrons to the required energy range. It is necessary to use the Monte Carlo method to model the actual situation and perform a large number of simulation calculations. This method has the following two problems:

[0006] (1) The time required for simulating and calculating the neutron energy spectrum is long and the efficiency is low. Once the moderator model is determined, if it is necessary to obtain the neutron energy spectrum of neutrons with other incident energies and directions emitted from the moderator exit window (such as an accelerator neutron source with changeable neutron energy), it is necessary to change the incident neutron energy and direction and re-simulate the calculation. Due to the limitation of the calculation speed of existing hardware, the calculation time is too long and all the work is repetitive, which is inefficient.

[0007] (2) Insufficient data accuracy. If you want to obtain the neutron energy spectrum after neutrons of different energies and directions are incident on the moderator within a certain energy range, calculating only a few energy points will result in insufficient data accuracy. For example, in the process of BNCT treatment planning, a high-precision neutron energy spectrum is required to prepare and evaluate the human body absorbed dose. Due to problem (1), if you want to improve data accuracy, you still need to consume a lot of computing time. Summary of the invention

[0008] The method introduced in the present invention is based on a deep neural network, a multi-layer perceptron, and takes the incident neutron energy and direction as features. The neutron energy spectrum obtained by the traditional Monte Carlo method is used as the true label, and a deep neural network model is established to predict the outgoing neutron energy spectrum. The time for obtaining the slowed-down neutron energy spectrum data of different incident energies and directions is greatly reduced. The neutron energy spectrum data after slowing down neutrons of any incident energy and direction is obtained, and the data accuracy of the energy spectrum is improved.

[0009] In order to achieve the above object, the present invention is implemented by adopting the following technical scheme: the method comprises:

[0010] Establishment of data set and data preprocessing: Use Monte Carlo simulation to obtain data set, perform feature extraction, normalization and noise reduction on the feature data in the data set;

[0011] Get the training set and test set;

[0012] Construct a deep neural network model; the deep neural network model is modified based on the multi-layer perceptron model, and the hidden layer in the multi-layer perceptron is replaced by a residual block;

[0013] Random mini-batch training: Set the hyperparameters of the deep neural network model, and train each training set and test set in batches, calculate its loss or accuracy, and continuously adjust the hyperparameters based on the calculation results to make the model converge;

[0014] Prediction: Use the trained model to predict the processed data, and obtain the moderated neutron energy spectrum under the conditions by inputting the required incident neutron energy and directionality.

[0015] In one embodiment, the feature extraction is to extract information that has a decisive influence on the prediction of neutron energy spectrum from the original data set obtained by Monte Carlo simulation as features to form feature data;

[0016] Normalization: After extracting the features, normalize them. Assuming the original feature value is x and the normalized feature value is y, the min-max normalization expression is: y = (x-min) / (max-min), where min and max are the minimum and maximum values ​​of feature x, respectively.

[0017] Noise reduction operation: Remove high-frequency noise by designing filters.

[0018] In one embodiment, the training set and the test set are obtained by dividing the energy spectrum data into three energy intervals of thermal neutrons, epithermal neutrons, and fast neutrons according to the physical characteristics of neutrons, and each energy interval is divided into a corresponding training set and a test set.

[0019] In one solution, the deep neural network model is modified based on a multi-layer perceptron model, and the hidden layer in the multi-layer perceptron is replaced with a residual block. In the residual block, a dropout layer is also added to prevent the model from overfitting.

[0020] In one embodiment, the random mini-batch training includes:

[0021] Set the hyperparameters of the deep neural network model: including learning rate, batch size, and number of iterations;

[0022] Training the model in batches: First, randomly shuffle the training set, and then divide the training set into multiple small batches according to the set batch size; each time, extract a batch of data from it, input it into the network for forward propagation, and calculate the loss function;

[0023] Calculate loss and back propagate: The loss function is used to quantify the error between the model's predicted energy spectrum and the actual energy spectrum;

[0024] Update model parameters: After calculating the gradient, use the gradient descent method or update the model parameters to reduce the loss;

[0025] Loop iteration: Repeat the above until the model loss converges on the training set or reaches the preset number of iterations.

[0026] Beneficial effects of the present invention:

[0027] (1) Reduce the time to obtain moderated neutron energy spectrum data: When the moderator structure is determined, the traditional method of obtaining the neutron energy spectrum after slowing down neutrons with different incident energies often requires a lot of time to perform Monte Carlo simulation. If you want to obtain the exit window energy spectrum with different incident energies and directions, the time cost of calculation may increase exponentially, and each calculation only changes the neutron incident energy but the time consumed remains basically unchanged, which is inefficient. The deep learning model based on massive data reduces the neutron energy spectrum prediction time from hours to seconds, greatly reducing the time cost of obtaining the neutron energy spectrum after slowing down neutrons with any incident energy.

[0028] (2) Improve the data accuracy of neutron energy spectrum: Since the traditional Monte Carlo method consumes a lot of computing time during the calculation process, the number of neutron energy spectra obtained after moderation is limited, and a large number of moderated neutron energy spectra generated by neutrons of other energies passing through the moderator will be missing, resulting in insufficient accuracy of neutron energy spectrum data, inaccurate calibration of neutron field instruments using certain energies, or inaccurate dose calculation during BNCT treatment. After the time cost of obtaining energy spectra is greatly reduced based on deep learning algorithms, the training model can be used to supplement the moderated neutron energy spectra with missing energies and directions, which will greatly expand the accuracy of the required data. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is a flow chart of the method of the present invention;

[0030] Figure 2 This is a schematic diagram of the model architecture;

[0031] Figure 3 Schematic diagram of residual block with and without fully connected layer (n2);

[0032] Figure 4 Schematic diagram of the moderator used to build the neutron spectrum data set;

[0033] Figure 5 Comparison diagram of the predicted energy spectrum and Monte Carlo calculated energy spectrum when the incident neutron energy is 9.8548 (MeV) and the neutron direction cosα=0.9;

[0034] Figure 6 Comparison between the predicted energy spectrum and the Monte Carlo calculated energy spectrum for an incident neutron energy of 12 (MeV) and cosα=-0.55 in the neutron direction. DETAILED DESCRIPTION

[0035] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Typical embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described in the present invention. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0036] Unless otherwise defined, all technical and scientific terms used in the present invention have the same meaning as those understood by those skilled in the art of the present invention. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0037] like Figure 1 As shown: A method for predicting the energy spectrum of slowed-down neutrons based on deep learning includes

[0038] S1. Data set establishment and data preprocessing: Monte Carlo simulation is used to obtain the data set, and feature extraction, normalization, and noise reduction operations are performed on the feature data in the data set.

[0039] Monte Carlo simulation calculation: According to the neutron transmission equation in the moderator, the statistical simulation method is used to calculate and obtain the state distribution of a large number of neutrons in the moderator to form the original data set. This kind of simulation calculation is generally performed using the Monte Carlo algorithm, and its basic principle is to use random sampling to solve mathematical problems that are difficult to calculate directly.

[0040] Feature extraction: For the original data set obtained by Monte Carlo simulation, the information that has a decisive influence on the prediction of the neutron energy spectrum is extracted as features, such as the energy and direction of the incident neutrons in the neutron source, to form characteristic data.

[0041] Normalization: After extracting features, they are usually normalized. Normalization is to scale the data range of the features to the same interval so that the weights between features are more balanced rather than dominated by some features with larger values. Assuming the original feature value is x and the normalized feature value is y, the min-max normalization expression is: y = (x-min) / (max-min), where min and max are the minimum and maximum values ​​of feature x, respectively.

[0042] Denoising: The results of Monte Carlo simulations are often affected by various noises, so denoising is required to improve the accuracy of predictions. There are many ways to reduce noise, such as by designing filters to remove high-frequency noise or by eliminating outliers to reduce the impact of noise on the model. Especially for tasks that use neural networks for prediction, denoising is a very important step, because neural networks can easily be overly sensitive to noise, which can affect the prediction results.

[0043] S2. Obtain training set and test set. Obtain training set and test set: According to the physical characteristics of neutrons, the energy spectrum data is divided into three energy intervals: thermal neutrons, epithermal neutrons, and fast neutrons. Each energy interval is divided into corresponding training set and test set.

[0044] The specific divisions are as follows:

[0045] Thermal neutron energy range: 0.01eV~0.5eV.

[0046] Epithermal neutron energy region: 0.5eV~10keV.

[0047] Fast neutron energy region: 10eV~12eV.

[0048] Next, we need to extract data from these three energy ranges from the entire data set.

[0049] For all thermal neutron data, we randomly select about 80% (this ratio is set according to requirements) as the training set, and the remaining 20% ​​as the test set.

[0050] Similarly, the data for epithermal neutrons and fast neutrons are divided into training sets and test sets in the same proportions.

[0051] In this way, we obtained three sets of training sets and test sets, each set corresponding to a different neutron energy range.

[0052] S3. Build a deep neural network model. The deep neural network model is modified based on the multi-layer perceptron model, and the hidden layer in the multi-layer perceptron is replaced by a residual block.

[0053] This method is a neutron spectrum prediction method based on deep learning, which relies on the design of deep neural network. The overall architecture of the deep neural network model is as follows: Figure 2 As shown, Figure 3 yes Figure 2 The dotted box shows the internal structure of the residual block.

[0054] The deep neural network model of this method is modified based on the multilayer perceptron (MLP) model. We replace the hidden layer in the multilayer perceptron with a residual block ( Figure 2 Compared with the previous basic model MLP, our deep neural network model is a multi-layer nested network structure, which makes the model not need to directly fit the ideal mapping from the original data x, but to fit the residual mapping that is easier to learn, and finally continuously approach the ideal mapping. The architecture of the residual block and related explanations are as follows Figure 3 shown.

[0055] In addition, we replaced the hidden layers in the multi-layer perceptron (MLP) with residual blocks. This transformation from "layers" to "blocks" allows the model to retain the information of real data while continuously expanding the learning depth of the model through the stacking of blocks, making the model show excellent generalization ability on different data sets. Overall model architecture Figure 2 shown.

[0056] In the residual block, we also add a dropout layer to prevent the model from overfitting. Dropout is a classic regularization method used to prevent the model from overfitting. Its basic principle can be found in Figure 3 And related explanation of dropout.

[0057] in:

[0058] Inputs represents the input features, namely the energy and direction of the proton.

[0059] The fully connected layer (N1, N2) represents a linear connection layer, where (N1, N2) represents the dimension of the linear connection layer. The mathematical principle is as follows:

[0060] L(x)=W1x+b1

[0061] ReLU (Rectified linear unit) stands for rectified linear unit, which is the activation function of the fully connected layer. Its mathematical principle is as follows:

[0062] ReLU(x)=max(0,L(x))

[0063] Outputs represents the prediction results, i.e. the distribution of neutron energy spectrum.

[0064] Resisual block means residual block, n means that the residual block can be connected to n layers, and the internal structure of Resisual block is as follows Figure 3 As shown:

[0065] in:

[0066] The Dropout layer is a regularization layer in a deep neural network, which is used to reduce the impact of model overfitting. Its basic mathematical principle is to discard each neuron in the hidden layer if the probability of its existence is p. The mathematical formula is as follows:

[0067]

[0068] Explanation of the residual block: Assume that the original input is x, and the ideal mapping you want to learn is f(x), as Figure 3 Finally, the input of the ReLU function is Figure 2 The part in the dashed box on the left needs to fit the mapping f(x) directly, while the part in the dashed box on the right needs to fit the residual mapping f(x)-x. The residual mapping is often easier to optimize in reality.

[0069] Through the above implementation steps, the invention has the following inventiveness compared with the prior art:

[0070] (1) Innovation of model architecture: The deep neural network model of this method is modified from the multilayer perceptron (MLP) model. We replace the hidden layer in the multilayer perceptron with a residual block ( Figure 1 Compared with the previous basic model MLP, our deep neural network model is a multi-layer nested network structure, which makes the model not need to directly fit the ideal mapping from the original data x, but to fit the residual mapping that is easier to learn, and finally continuously approach the ideal mapping.

[0071] In addition, we replaced the hidden layers in the multi-layer perceptron (MLP) with residual blocks. This transformation from "layers" to "blocks" allows the model to retain the information of real data while continuously expanding the learning depth of the model through the stacking of blocks, making the model show excellent generalization ability on different data sets. Overall model architecture Figure 1 shown.

[0072] In the residual block, we also add a dropout layer to prevent the model from overfitting. Dropout is a classic regularization method used to prevent the model from overfitting. Its basic principle can be found in Figure 3 And related explanation of dropout.

[0073] (2) A method based on deep learning to predict neutron energy spectrum (method innovation): The traditional method of obtaining a specific neutron energy spectrum requires the use of the Monte Carlo method to model the actual situation and perform a large number of simulation calculations. As mentioned above, this method has two problems: the time required to simulate and calculate the neutron energy spectrum is long and the efficiency is low, and the data accuracy of the simulation results is insufficient. We use a deep learning method to predict the neutron energy spectrum in specific situations to replace the Monte Carlo method, reduce the time to obtain the slowed neutron energy spectrum data and improve the data accuracy of the neutron energy spectrum, and use a new method to solve the traditional neutron energy spectrum prediction problem.

[0074] (Partition prediction improves prediction accuracy) At the same time, when using the deep neural network model to predict the neutron energy spectrum, we divide the energy spectrum data into three energy intervals: thermal neutrons, epithermal neutrons, and fast neutrons according to the physical characteristics of neutrons, and divide each energy interval into corresponding training sets and test sets. The prediction of the neutron energy spectrum in a specific energy interval based on data sets in different energy intervals can greatly improve the prediction accuracy compared to the overall prediction.

[0075] (3) Innovation in application scenarios: We apply the method of predicting the energy spectrum of moderated neutrons based on deep learning to boron neutron capture therapy (BNCT), apply AI to the field of nuclear medicine, and achieve cross-integration of multiple fields (artificial intelligence, nuclear physics, and biomedicine).

[0076] S4. Random small batch training: Set the hyperparameters of the deep neural network model, and train each training set and test set in batches, calculate its loss or accuracy, and continuously adjust the hyperparameters based on the calculation results to make the model converge.

[0077] Set the hyperparameters of the deep neural network model: These hyperparameters include learning rate (determines the speed of model learning, too large a value may cause the model to fail to converge, too small a value may make the training process too long), batch size (the number of samples used in each training), number of iterations (the number of times the entire training set is trained), etc. The setting of these hyperparameters is usually attributed to experience, and can also be automatically optimized through hyperparameter search methods such as grid search and Bayesian optimization.

[0078] Batch training model: First, randomly shuffle the training set, and then divide the training set into multiple small batches according to the set batch size. Each time, a batch of data is extracted and input into the network for forward propagation, and the loss function is calculated.

[0079] Calculate loss and back propagate: The loss function is used to quantify the error between the model's predicted energy spectrum and the actual energy spectrum. Commonly used loss functions include Mean Square Error (MSE) and Cross-Entropy Loss. After calculating the loss, the back propagation algorithm is used to calculate the gradient of each parameter.

[0080] Update model parameters: After calculating the gradient, use the gradient descent method or its improved versions (such as AdaGrad, RMSProp, Adam, etc.) to update the model parameters to achieve the purpose of reducing the loss.

[0081] Iteration: Repeat steps 2 to 4 above until the model loss converges on the training set or the preset number of iterations is reached.

[0082] S5. Prediction: Use the trained model to predict the processed data, and obtain the moderated neutron energy spectrum under the conditions by inputting the required incident neutron energy and directionality.

[0083] Embodiment 1

[0084] Get the moderator structure used by the data set as Figure 4As shown in the figure, the structure is an epithermal neutron moderation collimation structure. The main moderator materials are aluminum fluoride, calcium fluoride and titanium fluoride. Cadmium and bismuth absorb thermal neutrons and gamma rays. The inner layer of the collimator is lead and the outer layer is polyethylene. Monte Carlo calculated that neutrons with different incident energies and incident directions pass through the moderator. The incident neutron energy range is 0.01MeV to 12MeV, and the logarithmic interval is adopted. A total of 73 energy points are calculated. The incident neutron direction of each energy ranges from cosθ=0 to cosθ=1, and the interval is 0.05. A total of 21 angle points are calculated. Record the neutron energy spectrum of the exit window and establish a data set.

[0085] Table 1 is an example of hyperparameters of the deep neural network model (the table is for reference only and can be adjusted to obtain better prediction results). Apply the following hyperparameters to the above deep neural network model and run the script (reference for example running configuration: Windows 11 system, pytorch 11.9, python 3.8, Nvidia Geforce GTX 1650) to predict the neutron energy spectrum. Figure 5 (According to Monte Carlo calculation results, thermal neutrons account for 1.77%, epithermal neutrons account for 82.76%, and fast neutrons account for 15.47%) and Figure 6 (According to the Monte Carlo calculation results, the proportion of thermal neutrons: 2.98%, the proportion of epithermal neutrons: 92.55%, and the proportion of fast neutrons: 4.46%) are two examples of predicted results. The training time using a laptop is only about 40s. Limited by the hardware of the running equipment, if more advanced equipment is used, the time cost of obtaining the neutron energy spectrum will be further shortened. From the prediction results, the prediction accuracy of the neutron energy spectrum predicted by the deep neural network model in the epithermal neutron and fast neutron intervals is highly consistent with the prediction results of the traditional method, which provides the possibility of high-precision prediction of the neutron energy spectrum produced by neutrons with different incident energies in practice. Since the moderator forms an epithermal neutron field, the proportion of thermal neutrons in the radiation field is low and the statistics are poor, resulting in insufficient fitting of the prediction results of the deep neural network model with the prediction results of the traditional method as the true label in the thermal neutron area, but the overall impact on the energy spectrum is low.

[0086] Under this computer configuration, the time to predict a new energy spectrum based on the training model is less than 1 second, and the time required for Monte Carlo to calculate the number of 3×108 particles is 55 minutes (CPU: AMD 7950x), and the calculation time is reduced from hours to seconds. Based on this algorithm, the data accuracy can be further expanded on the basis of the original data set.

[0087] Table 1. Examples of hyperparameters for deep neural network models

[0088]

[0089] *True and False indicate whether to include the residual block of the fully connected layer.

[0090] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0091] It should be understood that the detailed description of the technical solutions of the present invention by means of the preferred embodiments is illustrative rather than restrictive. A person skilled in the art may modify the technical solutions described in the embodiments, or replace some of the technical features by equivalents, based on reading the specification of the present invention; and these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for predicting moderated neutron energy spectrum based on deep learning, characterized in that: The method includes: Establishment of data set and data preprocessing: Use Monte Carlo simulation to obtain data set, perform feature extraction, normalization and noise reduction on the feature data in the data set; Get the training set and test set; Construct a deep neural network model; the deep neural network model is modified based on the multi-layer perceptron model, and the hidden layer in the multi-layer perceptron is replaced by a residual block; Random mini-batch training: Set the hyperparameters of the deep neural network model, and train each training set and test set in batches, calculate its loss or accuracy, and continuously adjust the hyperparameters based on the calculation results to make the model converge; Prediction: Use the trained model to predict the processed data, and obtain the moderated neutron energy spectrum under the conditions by inputting the required incident neutron energy and direction.

2. The method for predicting moderated neutron energy spectrum based on deep learning according to claim 1, characterized in that: The feature extraction is for the original data set obtained by Monte Carlo simulation, and the information having a decisive influence on the prediction of neutron energy spectrum is extracted as a feature; Forming characteristic data; Normalization: After extracting the features, normalize them. Assuming the original feature value is x and the normalized feature value is y, the min-max normalization expression is: y = (x-min) / (max-min), where min and max are the minimum and maximum values ​​of feature x, respectively. Noise reduction operation: Remove high-frequency noise by designing filters.

3. The method for predicting moderated neutron energy spectrum based on deep learning according to claim 1, characterized in that: The acquisition of training sets and test sets divides the energy spectrum data into three energy intervals of thermal neutrons, epithermal neutrons, and fast neutrons according to the physical characteristics of neutrons, and each energy interval is divided into a corresponding training set and test set.

4. The method for predicting moderated neutron energy spectrum based on deep learning according to claim 1, characterized in that: The deep neural network model described in the invention is modified based on the multi-layer perceptron as a basic model, and the hidden layer in the multi-layer perceptron is replaced by a residual block. In the residual block, a dropout layer is also added to prevent the model from overfitting.

5. The method for predicting moderated neutron energy spectrum based on deep learning according to claim 1, characterized in that: The random mini-batch training includes: Set the hyperparameters of the deep neural network model: including learning rate, batch size, and number of iterations; Training the model in batches: First, randomly shuffle the training set data, and then divide the training set into multiple small batches according to the set batch size; extract a batch of data each time, input it into the network for forward propagation, and calculate the loss function; Calculate loss and back propagate: The loss function is used to quantify the error between the model's predicted energy spectrum and the actual energy spectrum; Update model parameters: After calculating the gradient, use the gradient descent method or update the model parameters to reduce the loss; Loop iteration: Repeat the above until the model loss converges on the training set or reaches the preset number of iterations.

Images