Nuclear power plant digital twin multi-parameter long-term online prediction and uncertainty analysis method

Abstract

A nuclear power plant digital twin multi-parameter long-term online prediction and uncertainty analysis method, in the offline stage, the system parameters of multiple inputs and multiple outputs are determined, after data acquisition and preprocessing, a training set is generated, then the network hyperparameter optimization is carried out through the Bayesian optimization algorithm, and then the optimized prediction network is trained; in the online stage, the trained prediction network is used for real-time long-term online prediction, at the same time, the prediction network is used for transfer learning, the criteria for online updating of the model are set from the aspects of time length and error threshold, and the calculation of the confidence interval is added in the updating process, the prediction performance of the model is judged from the perspective of interval prediction, and the online updating of the model is realized. The present application realizes long-term prediction of multiple parameters and multiple steps; considering the operation system of the nuclear power plant, systematic prediction is carried out; combining transfer learning and online updating, the generalization ability of the model is proved; the uncertainty problem is considered; a complete time series prediction system is formed.

Description

Technical Field

[0001] This invention relates to a technology in the field of nuclear power plant control, specifically a method for long-term online prediction and uncertainty analysis of multiple parameters in a nuclear power plant digital twin. Background Technology

[0002] Nuclear power plants suffer from low utilization of actual data, and this invention addresses this issue through in-depth data mining. Meanwhile, time-series forecasting models face challenges such as lag, forgetting, and poor accuracy in long-term predictions, as well as issues related to handling large amounts of input data and poor test model performance. Summary of the Invention

[0003] This invention addresses the shortcomings of existing prediction technologies, which employ RNNs and LSTMs for short-term predictions of single components under single operating conditions. These methods suffer from high computational complexity, low accuracy, reliance on expert systems, and inability to handle long-term predictions with multiple inputs and outputs. The invention proposes a digital twin-based multi-parameter long-term online prediction and uncertainty analysis method for nuclear power plants. This method fully explores the potential relationships between data. Based on the characteristics of time-series data, it improves existing models at the algorithmic level, achieving coupled multi-parameter prediction, multi-step long-term prediction, and systematic prediction considering the nuclear power plant's operating system. By combining transfer learning and online updates, the generalization ability of the model is demonstrated. Uncertainty issues are considered, and a complete time-series prediction system is formed.

[0004] This invention is achieved through the following technical solution:

[0005] This invention relates to a method for long-term online prediction and uncertainty analysis of multi-parameter digital twins for nuclear power plants. In the offline phase, multi-input multi-output system parameters are determined. After data acquisition and preprocessing to generate a training set, the network hyperparameters are first optimized using a Bayesian optimization algorithm, and then the optimized prediction network is trained. In the online phase, while using the trained prediction network for real-time long-term online prediction, transfer learning is performed on the prediction network. Criteria for online model updates are set from two perspectives: time length and error threshold. Confidence interval calculations are incorporated into the update process to judge the model's predictive performance from the perspective of interval prediction, thus achieving online model updates.

[0006] The aforementioned digital twin for nuclear power plants refers to the interactive mapping between a digital model constructed in virtual space and a physical entity, using historical data, real-time data, and algorithm models to simulate, verify, predict, and control the operation of the physical entity throughout its entire lifecycle.

[0007] The prediction network comprises: a position encoding layer, a data feature upscaling layer, an encoder, a decoder, and a linear layer. Specifically: the position encoding layer encodes the input sequence using the tanh function to obtain a sequence representation containing position information; the data feature upscaling layer processes the input sequence's features using a linear layer to obtain an upscaled feature representation; the encoder processes the upscaled feature representation using a self-attention mechanism and a feedforward neural network to obtain an encoded sequence representation; the decoder processes the encoded sequence representation and the target sequence's position information using a self-attention mechanism, an encoder-decoder attention mechanism, and a feedforward neural network to obtain a decoded sequence representation; and the fully connected layer performs a linear transformation on the decoded sequence representation to obtain the final output.

[0008] The aforementioned network hyperparameter optimization specifically includes:

[0009] 1) Determine the optimization objective, select the loss value to represent the model's performance index, and set the optimization objective to guide the direction of hyperparameter adjustment;

[0010] 2) Using the learning rate, batch size, network depth, and width as hyperparameters to be optimized, the hyperparameters are gradually adjusted and the model is trained using the Bayesian optimization method. The effect of each adjustment is evaluated based on the performance on the validation set until the optimal combination of hyperparameters that meets the optimization objective is found.

[0011] The aforementioned online updates specifically include:

[0012] i) Real-time data collection, continuously collecting new data to ensure the real-time nature and accuracy of the data stream;

[0013] ii) Incremental training: Based on the new data collected, the model parameters are updated step by step using incremental training methods to avoid training from scratch and improve the efficiency and effectiveness of model updates;

[0014] iii) Model monitoring and evaluation: Monitor the performance of the updated model in real-time in practical applications, evaluate model performance regularly to ensure that the model always remains in the best state, and make further adjustments and optimizations based on error thresholds and set time intervals. Technical effect

[0015] This invention addresses the non-periodic and non-linear characteristics of nuclear power plant data. Building upon the existing Transformer model, it proposes an improved time-series prediction model for nuclear power plant data. This model employs dimensionality enhancement of the time-series data to extract features in a higher-dimensional space. Considering that the closer the time-series data is to the prediction point, the greater its impact, a monotonically increasing function is used to assign weights to the input data. This achieves multi-parameter, multi-step, long-term time-series prediction. Through transfer learning, this invention verifies the generalization performance and robustness of the trained model on other samples. Furthermore, since time series data is a typical type of streaming data that changes over time, the uneven distribution of input data can lead to poor model generalization ability. This invention introduces an incremental learning algorithm to update the model weights after a certain period, effectively improving the model's generalization ability.

[0016] Compared to existing technologies, this invention addresses the non-periodic and non-linear characteristics of nuclear power plant data. Building upon the existing Transformer model, it proposes a time-series prediction model for nuclear power plant data. This model improves upon the existing Transformer model by performing dimensionality upscaling on the time-series data, extracting data features in a higher-dimensional space. Considering that the closer the time-series data is to the prediction point, the greater its influence, a monotonically increasing function is used to assign weights to the input data. This achieves multi-parameter, multi-step, long-term time-series prediction. Furthermore, the data used in this invention are all actual data generated from the daily operation of a nuclear power plant, ensuring data reliability and authenticity. The final model exhibits good prediction performance and high accuracy. Through transfer learning, this invention verifies the generalization performance and robustness of the trained model on other samples. Moreover, time series data is a typical type of streaming data that changes over time. As time increases, the uneven distribution of input data can lead to poor model generalization ability. This invention introduces an incremental learning algorithm to update the model weights after a certain period, effectively improving the model's generalization ability. This invention also considers measurement errors and model uncertainties during actual operation, proposing a dynamically updated confidence interval calculation method based on Bayesian methods to evaluate model performance. Attached Figure Description

[0017] Figure 1 This is a flowchart of the present invention;

[0018] Figure 2 A schematic diagram of the prediction network;

[0019] Figure 3 Loss plot of the pre-training model after hyperparameter optimization;

[0020] Figure 4 The improved Transformer's time series prediction results;

[0021] In the figure: ag represents the comparison curves of predicted and actual values ​​of the following data in the first group: hot pipe section temperature, cold pipe section temperature, primary loop coolant temperature, pressurizer pressure, pressurizer water level, steam generator secondary side feedwater flow rate, and nuclear power.

[0022] Figure 5 This is a graph showing the results of transfer learning.

[0023] In the figure: ag represents the comparison curves between the predicted and actual values ​​of the following data in the second set: hot pipe section temperature, cold pipe section temperature, primary loop coolant temperature, pressurizer pressure, pressurizer water level, steam generator secondary side feedwater flow rate, and nuclear power.

[0024] Figure 6 To update the results image online;

[0025] Figure 7 Bootstrap flowchart;

[0026] Figure 8 The confidence interval range for uncertainty analysis;

[0027] In the figure: ag represents the confidence intervals of the hot pipe section temperature, cold pipe section temperature, primary loop coolant temperature, pressurizer pressure, pressurizer water level, secondary side feedwater flow rate of steam generator, and nuclear power in the first set of data. Detailed Implementation

[0028] like Figure 1 As shown in the figure, this embodiment relates to a method for long-term online prediction and uncertainty analysis of multiple parameters in a nuclear power plant digital twin, specifically including:

[0029] Step 1: Input data preprocessing, specifically including:

[0030] 1.1 Information mining of input data: The data collected in different time intervals and the water flow measurement data are smoothed by using a low-pass filter, and the selected system parameters are normalized to their maximum and minimum values.

[0031] The collected data consists of data from 123 measuring points in the nuclear power plant's operating system, including primary loop reactor core, hot and cold pipe sections, pressure vessel, steam generator, secondary loop key parameters, chemical and volumetric system parameters, and control system parameters.

[0032] The selected system parameters include seven key system operating parameters: hot pipe section temperature, cold pipe section temperature, primary loop coolant temperature, pressurizer pressure, pressurizer water level, secondary side feedwater flow rate of steam generator, and nuclear power.

[0033] 1.2 Select 30 time steps as a group of inputs. By inputting the first 30 time steps, predict the time series data for the next 30 time steps. At the same time, group the data of the same time step of the selected 7 parameters into a group. Divide the grouped data into training set, validation set and test set in a ratio of 2:1:7.

[0034] Step 2: Improve and optimize the Transformer model, specifically including:

[0035] 2.1 Based on the Transformer model, a linear layer is added to perform digital transformation of information: In time series prediction, a fully connected layer is used to increase the feature quantity of the data from 1 dimension to a high dimension, making the feature structure sparse.

[0036] 2.2 Define the position encoding function x' = x + k * PE, where: x is the preprocessed data, x' is the data after adding position encoding, k is the scaling factor, PE is the position encoding function, and PE ∈ [-1, 1].

[0037] In this embodiment, the monotonically increasing hyperbolic tangent function tanh is selected as the position encoding function PE, and the scaling factor k is set to 1.

[0038] 2.3 Further training of the encoder, decoder, and self-attention mechanism is added to form the prediction network of this invention. Then, all parameters of the training set obtained in step 1 are simultaneously input into the network. Figure 2 The prediction network shown.

[0039] Step 3: Optimize the network's hyperparameters using Bayesian hyperparameter optimization, specifically including:

[0040] 3.1 Determine the optimization objective. Based on the model's performance metrics, set the loss value as the optimization objective to guide the direction of hyperparameter adjustment. Set the optimization objective function L(θ), where θ represents the set of hyperparameters to be optimized.

[0041] 3.2 Define the hyperparameter space. Based on model requirements and prior knowledge, define the search space for learning rate, batch size, number of network layers, and depth. Let the hyperparameter vector be θ = (θ1, θ2, ..., θ...). n Each hyperparameter has a value range of θ. i ∈[a i ,b i ]

[0042] 3.3 Iterative optimization: Through continuous sampling and evaluation, the optimal hyperparameter combination is gradually approached until the preset optimization objective is met or the maximum number of iterations is reached. Initial sampling involves randomly sampling a set of hyperparameters θ within the defined hyperparameter space. (0) The model is trained based on the sampling hyperparameters, and the loss value L(θ) is calculated. (0)Based on the initial sampling results, the posterior distribution of hyperparameters is updated using Bayesian optimization. Then, a new combination of hyperparameters θ is selected based on the updated posterior distribution. (t+1) Perform the evaluation, repeating the above steps until the loss value L(θ) is reached. (t) It converges to the preset threshold or reaches the maximum number of iterations.

[0043] 3.4 Validation and Application: The final model is trained using the found optimal hyperparameter combination, and its performance is evaluated on an independent validation set to ensure the reliability and generalization ability of the optimization effect. The error between the predicted values ​​and actual values ​​on the training set is calculated using the mean absolute error (MAE) to judge the performance of the trained time series prediction model. Specifically: Where: x i For the true value, y i is the actual value, and n is the number of samples.

[0044] like Figure 3 The figure shown is a graph of the pre-trained time series prediction model after hyperparameter optimization. As can be seen from the figure, the loss basically converges after 3000 training iterations.

[0045] like Figure 4 The figure shown is a graph illustrating the results of multi-step, multi-parameter prediction. It can be seen that the predicted curve fits the actual values ​​well. This result effectively demonstrates the good performance of the improved Transformer long-term prediction model.

[0046] Step 4, Transfer Learning and Online Update: Save the network hyperparameters and model weights of the pre-trained model after hyperparameter optimization using the first set of data, and transfer them to the second set of actual nuclear power plant data. Use the previously learned feature knowledge to perform multi-step time series prediction on the second set of data.

[0047] This embodiment employs a fine-tuning method, adjusting the last fully connected layer with a relatively small learning rate. The trained transfer model is updated online using 60 time steps and an error threshold as update metrics, achieving the following results: Figure 6 As shown in the figure, the yellow dashed line represents the prediction performance of the pre-trained model. It can be seen that by fine-tuning the weights, the model can learn the distribution of new data and fit the true value well.

[0048] Technical effect evaluation

[0049] a) such as Figure 7 As shown, the Bootstrap method is used to estimate the standard error, confidence interval, and bias through resampling techniques, and the normal distribution of small sample data is calculated.

[0050] b) Perform Bayesian prior and posterior analysis on the normally distributed data from step a, specifically: assuming the sample in: It is the mean of the prior distribution. It is the standard deviation of the prior distribution.

[0051] c) The parameter values ​​of the maximum likelihood probability are derived from the premise that the Bayesian prior distribution based on the known observation data is obtained through maximum likelihood estimation. Specifically: in: is the sample mean, and n is the sample size.

[0052] d) Update the posterior probability after observing the data, specifically: Where: y m For observational data.

[0053] Through specific practical experiments, running the above device in the Python PyTorch environment, the experimental data that can be obtained are: the time series prediction results of multi-step and multi-input nuclear power plant operation data for the first and second groups.

[0054] Compared with existing technologies, this invention uses a linear layer to increase the dimensionality of data and performs deep data mining, solving the problem of low utilization of actual data due to the large amount of accumulated data in nuclear power plants and the failure to fully extract effective information from real data. Addressing the non-periodic and non-linear characteristics of time-series data, the function at the Transformer position encoding is improved, making this method more suitable for long-term prediction models of time-series forecasting and achieving higher prediction accuracy compared to conventional Transformer models. This invention also implements transfer learning and online updates, demonstrating not only better generalization ability of the proposed model but also identifying new data stream distribution patterns through online update technology, thus improving the model's prediction accuracy. Finally, based on Bayesian theory, this invention proposes a dynamic update uncertainty analysis method, quantifying the model's errors and providing a better assessment of model uncertainty.

[0055] The above-described specific implementations can be partially adjusted by those skilled in the art in different ways without departing from the principles and purpose of the present invention. The scope of protection of the present invention is defined by the claims and is not limited to the above-described specific implementations. All implementation schemes within the scope of the claims are bound by the present invention.

Claims

1. A method for long-term online prediction and uncertainty analysis of multiple parameters in a digital twin of a nuclear power plant, characterized in that, In the offline phase, the system parameters for multiple inputs and multiple outputs are determined. After data acquisition and preprocessing to generate a training set, the network hyperparameters are first optimized using a Bayesian optimization algorithm, and then the optimized prediction network is trained. In the online phase, the trained prediction network is used for real-time long-term online prediction while simultaneously performing transfer learning. Criteria for online model updates are set from the perspectives of time length and error threshold, and confidence interval calculations are incorporated into the update process to evaluate the model's predictive performance from the perspective of interval prediction, thus achieving online model updates. Specifically, this includes: Step 1: Input data preprocessing, specifically including: 1.1 Information mining of input data: The data collected in different time intervals and the water flow measurement data are smoothed by using a low-pass filter, and the selected system parameters are normalized by maximum and minimum values. The collected data consists of data from 123 measuring points in the nuclear power plant's operating system, including primary loop reactor core, hot and cold pipe sections, pressure vessel, steam generator, secondary loop key parameters, chemical and volumetric system parameters, and control system parameters. The selected system parameters include seven key system operating parameters: heat pipe section temperature, cold pipe section temperature, primary loop coolant temperature, pressurizer pressure, pressurizer water level, steam generator secondary side feedwater flow rate, and nuclear power. 1.2 Select 30 time steps as a group of inputs. By inputting the first 30 time steps, predict the time series data of the next 30 time steps. At the same time, group the data of the same time step of the selected 7 parameters into a group. Divide the grouped data into training set, validation set and test set in a ratio of 2:1:

7. Step 2: Improve and optimize the Transformer model, specifically including: 2.1 Based on the Transformer model, a linear layer is added to perform digital transformation of information: In time series prediction, a fully connected layer is used to increase the feature quantity of the data from 1 dimension to a high dimension, making the feature structure sparser. 2.2 Define the position encoding function ,in: For the preprocessed data, This represents the data after location encoding, where k is the scaling factor and PE is the location encoding function. We choose the monotonically increasing hyperbolic tangent function tanh as the position encoding function PE, and set the scaling factor k to 1. 2.3 Further training of the encoder, decoder, and self-attention mechanism is added to form the prediction network. Then, all parameters from the training set obtained in step 1 are simultaneously input into the prediction network. Step 3: Optimize the network's hyperparameters using Bayesian hyperparameter optimization, specifically including: 3.1 Define the optimization objective: Based on the model's performance metrics, set the loss value as the optimization objective to guide the direction of hyperparameter adjustment, and define the optimization objective function. ,in This represents the set of hyperparameters to be optimized. 3.2 Define the hyperparameter space. Based on model requirements and prior knowledge, define the search space for learning rate, batch size, number of network layers, and depth. Let the hyperparameter vector be... The range of values ​​for each hyperparameter is: ; 3.3 Iterative optimization: Through continuous sampling and evaluation, the optimal hyperparameter combination is gradually approached until the preset optimization objective is met or the maximum number of iterations is reached. The initial sampling involves randomly sampling a set of hyperparameters within the defined hyperparameter space. The model is trained based on the sampling hyperparameters, and the loss value is calculated. Based on the initial sampling results, the posterior distribution of hyperparameters is updated using Bayesian optimization. Then, a new combination of hyperparameters is selected based on the updated posterior distribution. Perform an evaluation, repeating the above steps until the loss value is reached. Converging to a preset threshold or reaching the maximum number of iterations; 3.4 Validation and Application: The final model is trained using the found optimal hyperparameter combination, and its performance is evaluated on an independent validation set to ensure the reliability and generalization ability of the optimization effect. The error between the predicted values ​​and actual values ​​on the training set is calculated using the mean absolute error (MAE) to judge the performance of the trained time series prediction model. Specifically: ,in: For the true value, The actual value is n, where n is the number of samples; Step 4, Transfer Learning and Online Update: Save the network hyperparameters and model weights of the pre-trained model after hyperparameter optimization using the first set of data, and transfer them to the second set of actual nuclear power plant data. Use the previously learned feature knowledge to perform multi-step time series prediction on the second set of data.

2. The method for long-term online prediction and uncertainty analysis of multi-parameter digital twins for nuclear power plants according to claim 1, characterized in that, The aforementioned digital twin of a nuclear power plant refers to the interactive mapping between a digital model constructed in a virtual space and a physical entity, using historical data, real-time data, and algorithm models to simulate, verify, predict, and control the operation of the physical entity throughout its entire life cycle.

3. The method for long-term online prediction and uncertainty analysis of multi-parameter digital twins for nuclear power plants according to claim 1, characterized in that, The prediction network comprises: a position encoding layer, a data feature upscaling layer, an encoder, a decoder, and a linear layer. Specifically: the position encoding layer encodes the input sequence using the tanh function to obtain a sequence representation containing position information; the data feature upscaling layer processes the input sequence's features using a linear layer to obtain an upscaled feature representation; the encoder processes the upscaled feature representation using a self-attention mechanism and a feedforward neural network to obtain an encoded sequence representation; the decoder processes the encoded sequence representation and the target sequence's position information using a self-attention mechanism, an encoder-decoder attention mechanism, and a feedforward neural network to obtain a decoded sequence representation; and the fully connected layer performs a linear transformation on the decoded sequence representation to obtain the final output.

4. The method for long-term online prediction and uncertainty analysis of multi-parameter digital twins for nuclear power plants according to claim 1, characterized in that, The aforementioned network hyperparameter optimization specifically includes: 1) Determine the optimization objective, select the loss value to represent the model's performance index, and set the optimization objective to guide the direction of hyperparameter adjustment; 2) Using the learning rate, batch size, network depth, and width as hyperparameters to be optimized, the hyperparameters are gradually adjusted and the model is trained using the Bayesian optimization method. The effect of each adjustment is evaluated based on the performance on the validation set until the optimal combination of hyperparameters that meets the optimization objective is found.

5. The method for long-term online prediction and uncertainty analysis of multi-parameter digital twins for nuclear power plants according to claim 1, characterized in that, The aforementioned online updates specifically include: i) Real-time data collection, continuously collecting new data to ensure the real-time nature and accuracy of the data stream; ii) Incremental training: Based on the new data collected, the model parameters are updated step by step using incremental training methods to avoid training from scratch and improve the efficiency and effectiveness of model updates; iii) Model monitoring and evaluation: Monitor the performance of the updated model in real-time in practical applications, evaluate model performance regularly to ensure that the model always remains in the best state, and make further adjustments and optimizations based on error thresholds and set time intervals.

Images