Nuclear power safety radiation dose rate predictor based on attention decomposition mechanism

Abstract

The invention discloses a nuclear power safety radiation dose rate predictor based on an attention decomposition mechanism, which comprises a database, an upper computer and a sensor, and the database, the upper computer and the sensor are connected in sequence. The database comprises radiation dose rate data of nuclear power safety history monitoring. The upper computer comprises a supervision training module, a data prediction module and a prediction display module. The supervision training module uses nuclear power safety historical monitoring data to train a depth prediction network model and transmits the depth prediction network model to the data prediction module. The sensor obtains nuclear power safety monitoring data in real time and transmits the nuclear power safety monitoring data to the data prediction module of the upper computer for prediction and estimation, and a prediction result is displayed on the prediction display module of the upper computer. According to the invention, the attention decomposition mechanism is innovatively used to obtain the season and trend characteristics of the data, so that the inherent characteristics and rules of the data are better grasped, and the nuclear power safety radiation dose rate predictor based on the attention decomposition mechanism is high in accuracy and capable of realizing online prediction.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and particularly to a nuclear power safety radiation dose rate predictor based on an attention decomposition mechanism. Background Art

[0002] The prediction of nuclear power safety radiation dose rate is a key technology, which is widely applied in fields such as the nuclear energy industry, nuclear accident emergency management, and environmental protection. Accurately predicting the nuclear power safety radiation dose rate is of great significance, which can be used to ensure people's health and environmental safety and provide decision-making support. However, due to the complexity and uncertainty of nuclear power safety, the prediction of nuclear radiation dose rate faces various challenges. Firstly, nuclear power safety is affected by multiple factors, such as nuclear accidents, weather conditions, environmental factors, etc. The changes of these factors will lead to the instability and volatility of radiation dose rate data. Secondly, the monitoring data of nuclear power safety may be incomplete and inaccurate, which may be caused by limitations of monitoring equipment, difficulties in data collection, and errors in monitoring data processing. Incomplete and inaccurate data will affect the establishment of the prediction model and the accuracy of the prediction results.

[0003] Currently, there have been some research works on the prediction of nuclear power safety radiation dose rate, mainly including statistical and model-based methods, machine learning and artificial intelligence-based methods, and time series analysis-based methods. However, these methods have some common limitations in the prediction of nuclear power safety radiation dose rate: on the one hand, they cannot effectively process the seasonal and trend features in the data, which limits the accuracy and stability of the prediction model; on the other hand, they have insufficient understanding and cumbersome processing of the complex features and rules of nuclear power safety data, resulting in the reliability and real-time performance of the prediction model needing to be improved. Therefore, there is an urgent need to develop a new type of nuclear power safety radiation dose rate predictor to overcome the deficiencies of the existing methods and improve the accuracy and real-time performance of the prediction. This instrument should have the ability to effectively process and analyze nuclear power safety data, be able to make full use of historical monitoring data and real-time monitoring data, extract key features and make accurate predictions. At the same time, this instrument should also have good adaptability and scalability, be able to cope with different environmental conditions and monitoring requirements, and provide reliable radiation dose rate prediction services for the nuclear energy industry and environmental protection. The innovation of the present invention lies in introducing an attention decomposition mechanism, through which the seasonal and trend features of the data can be better captured, the accuracy and real-time performance of the predictor can be improved, and a new solution is provided for the prediction of nuclear power safety radiation dose rate. Summary of the Invention

[0004] In order to overcome the deficiencies of the existing nuclear power safety radiation dose rate predictors, such as poor accuracy and weak real-time performance, the purpose of the present invention is to provide a nuclear power safety radiation dose rate predictor based on an attention decomposition mechanism, which can make full use of the data laws of historical monitoring data and real-time monitoring data, efficiently extract potential features in the data, and achieve reliable, accurate, and timely prediction results.

[0005] The technical solution adopted by the present invention to solve its technical problems is: a nuclear power safety radiation dose rate predictor based on an attention decomposition mechanism, including a database, a host computer, and sensors, which are connected in sequence; the database is used to store the radiation dose rate data of historical nuclear power safety monitoring, and the data stored in the database is used by the host computer; the host computer includes a supervised training module, a data prediction module, and a prediction display module that are connected in sequence; the supervised training module of the host computer uses the radiation dose rate data of historical nuclear power safety monitoring to train a deep prediction network model, and then uploads the trained deep prediction network model to the data prediction module of the host computer; the sensor obtains nuclear environment monitoring data in real time and uploads it to the data prediction module of the host computer, and the data prediction module makes a prediction estimate of the real-time nuclear environment monitoring data; finally, the prediction result is displayed on the prediction display module of the host computer.

[0006] Further, given the radiation dose rate data of historical nuclear power safety monitoring, abbreviated as historical monitoring data X t =[x t+1 ,x t+2 ,...,x t+T , where x t+1 represents the historical monitoring data at the (t + 1)-th time step, T represents the length of the historical monitoring data time window, where D represents the dimension of the historical monitoring data; is the radiation dose rate from the (t + T + 1)-th to the (t + T + H)-th time step, H represents the length of the prediction data time window; the goal is to learn a function such that Further, the supervised training module includes a deep prediction network embedding layer module, a deep prediction network encoder module, a deep prediction network decoder module, and a deep prediction network training module:

[0007] 1) Deep prediction network embedding layer module, the deep prediction network embedding layer module performs embedding encoding processing on the radiation dose rate data of historical nuclear power safety monitoring, encodes the data using multiple embedding forms, better grasps the internal characteristics and laws of the data, and improves prediction accuracy. It includes position embedding, token embedding, and time embedding:

[0008] X em=Position(X in )+Token(X in )+Temporal(X in ) (1)

[0009] where X in =X t represents the input radiation dose rate data for nuclear power safety historical monitoring, Position(·), Token(·), and Temporal(·) respectively represent position embedding, token embedding, and temporal embedding operations, and X em represents the output of the deep prediction network embedding layer module.

[0010] 2) The deep prediction network encoder module efficiently extracts data features to further improve prediction accuracy. The deep prediction network encoder module encodes the output of the deep prediction network embedding layer module for feature extraction. The specific process is as follows:

[0011] First, the multi-head attention of the sequence is expressed as:

[0012] MultiHead(Q, K, V)=Concat(h 1 , h 2 , …, h n )W (2)

[0013] h i =Attention(Q i , K i , V i ) (3)

[0014]

[0015] where MultiHead(·) represents the multi-head attention operation, Concat(·) represents the concatenation operation, Attention(·) represents the attention operation, SoftMax(·) represents the normalization operation on the attention scores, Q, K, and V represent the query, key, and value matrices, h i represents the i-th attention head, 1 ≤ i ≤ n, n represents the total number of attention heads, W represents the mapping matrix, Q i , K i , and V i represent the i-th query, key, and value matrices, and d k represents the first dimension of the Q i matrix. Perform multi-head attention extraction on the output of the deep prediction network embedding layer module:

[0016] X a =MultiHead(X em) (5)

[0017] Among them, X a represents the encoded output after multi-head attention.

[0018] Next, decompose the output of the multi-head attention layer, and define the module for decomposing the output of the multi-head attention layer as the decomposer. The decomposer performs a padding operation on the input before the feature extraction step, and then passes through a one-dimensional average pooling layer. The average pooling layer is used to traverse the entire data time series and find its seasonal and trend features. Convolution kernels of different sizes extract seasonal and trend features of different lengths, better grasping the internal features and patterns of the data and improving the prediction accuracy. Obtain the final seasonal and trend terms:

[0019]

[0020] Among them, Decomposer(·) represents the decomposer operation, and respectively represent the seasonal and trend feature terms of the data in the encoder.

[0021] Then, use convolutional layers to learn the seasonal and trend feature terms of the encoder respectively. The output of the convolutional layer passes through the decomposer again to further reveal the temporal correlation of the data:

[0022]

[0023] Among them, and respectively represent the seasonal and trend feature terms of the data in the encoder after feature extraction.

[0024] Finally, the output of the deep prediction network encoder module is obtained by the feed-forward layer:

[0025]

[0026] Among them, LayerNorm(·) represents the layer normalization operation, Linear(·) represents the multi-layer perceptron, and X en represents the output of the deep prediction network encoder module.

[0027] 3) Deep prediction network decoder module, the deep prediction network decoder module decodes the output of the deep prediction network encoder module and obtains the final prediction output. The specific process is as follows:

[0028] First, the original data, that is, the detection data, directly enters the decoder without embedding processing. The first module of the decoder is the decomposer:

[0029]

[0030] Among them, and respectively represent the seasonal and trend feature terms of the data in the decoder, and X de = X t represents the input data of the decoder, that is, the original data.

[0031] Next, use the convolutional layer to separately learn the seasonal and trend feature terms of the decoder to obtain the seasonal and trend feature terms in the decoder after feature extraction and

[0032]

[0033] Finally, the output of the deep network encoder module is summed with the seasonal and trend feature terms in the decoder after feature extraction and input into the feed-forward layer to obtain the decoder output:

[0034]

[0035] wherein, is the radiation dose rate from the (t + T + 1)-th to the (t + T + H)-th time step predicted by the network, α represents the output proportion factor of the output X en of the deep prediction network encoder module.

[0036] 4) Deep prediction network training module, the deep prediction network training module is used to train the deep prediction network model, and the training objective is:

[0037]

[0038] wherein, represents the loss function, and the backpropagation is used to train the model during the training process.

[0039] 5) Upload the trained deep prediction network model to the data prediction module.

[0040] Furthermore, the sensor acquires the radiation dose rate data of nuclear environment monitoring in real time and uploads it to the data prediction module. The data prediction module receives the trained deep prediction network model transmitted from the supervised training module and uses this model to predict and estimate the real-time monitoring data; finally, the prediction result is displayed on the prediction display module.

[0041] The beneficial effects of the present invention are as follows: For the problem of predicting the nuclear power safety radiation dose rate, the present invention encodes data in various embedding forms, uses an encoder and a decoder based on the attention mechanism to efficiently extract data features and make predictions, and adopts an attention decomposition mechanism to obtain the seasonal and trend features of the data, better grasping the internal features and laws of the data, and finally realizing real-time intelligent prediction of nuclear power safety. It has the following advantages: 1. High prediction accuracy and strong stability; 2. Strong real-time performance and high credibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0043] Figure 1 It is a schematic diagram of the hardware structure of the instrument proposed by the present invention;

[0044] Figure 2 It is a schematic diagram of the functional modules of the host computer proposed by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0045] The present invention will be described in detail below with reference to the drawings. Without conflict, the features in the following embodiments and implementation manners can be combined with each other.

[0046] Refer to Figure 1 、 Figure 2 , a nuclear power safety radiation dose rate predictor based on an attention decomposition mechanism, including a database 1, a host computer 2 and a sensor 3, which are connected in sequence; the radiation dose rate data of historical nuclear power safety monitoring is stored in the database 1, and the data stored in the database 1 is used by the host computer 2; the host computer 2 includes a supervision and training module 4, a data prediction module 5 and a prediction display module 6 connected in sequence; the supervision and training module 4 of the host computer 2 uses the historical nuclear power safety monitoring data to train a deep prediction network model, and then uploads the trained deep prediction network model to the data prediction module 5 of the host computer; the sensor 3 acquires nuclear environment monitoring data in real time and uploads it to the data prediction module 5 of the host computer 2, and the data prediction module 5 uses the trained deep prediction network model to predict and estimate the real-time monitoring data; finally, the prediction result is displayed on the prediction display module 6 of the host computer 2.

[0047] Specifically, the database 1 is responsible for storing a large amount of radiation dose rate data of historical nuclear power safety monitoring. Given the historical monitoring data X t =[x t+1 ,xt+2 , …, x t+T , where x t+1 represents the historical monitoring data at the (t + 1)-th time step, and T represents the length of the time window of the historical monitoring data, where D represents the dimension of the historical monitoring data; is the radiation dose rate from the (t + T + 1)-th to the (t + T + H)-th time step, and H represents the length of the time window of the predicted data; The goal is to learn a function such that

[0048] Specifically, the supervised training module 4 includes a deep prediction network embedding layer module 7, a deep prediction network encoder module 8, a deep prediction network decoder module 9, and a deep prediction network training module 10:

[0049] 1) The deep prediction network embedding layer module 7 embeds and encodes the original data of nuclear power safety monitoring, encodes the data using multiple embedding forms, better grasps the internal characteristics and laws of the data, and improves the prediction accuracy. It includes positional embedding, token embedding, and temporal embedding. Positional embedding encodes the input order of the data using sine and cosine functions; Token embedding is a one-dimensional convolution operation to expand the feature dimension of the data; Temporal embedding is responsible for further extracting the temporal relationship between the data:

[0050] X em = Position(X in ) + Token(X in ) + Temporal(X in ) (1)

[0051] where X in = X t represents the original data of nuclear power safety monitoring as input, Position(·), Token(·), and Temporal(·) represent positional embedding, token embedding, and temporal embedding operations respectively, and X em represents the output of the deep prediction network embedding layer module 7.

[0052] 2) The deep prediction network encoder module 8 efficiently extracts data features and further improves the prediction accuracy. The deep prediction network encoder module 8 encodes the output of the deep prediction network embedding layer module 7 for feature extraction. The specific process is as follows:

[0053] First, the multi-head attention of the sequence is expressed as:

[0054] MultiHead(Q, K, V) = Concat(h 1 , h2 ,…,h n )W(2)

[0055] h i =Attention(Q i ,K i ,V i ) (3)

[0056]

[0057] where MultiHead(·) represents the multi - head attention operation, Concat(·) represents the concatenation operation, Attention(·) represents the attention operation, SoftMax(·) represents the normalization operation on attention scores, Q, K, and V represent the query, key, and value matrices, h i represents the i - th attention head, n represents the total number of attention heads, W represents the mapping matrix, Q i , K i and V i represent the i - th query, key, and value matrices, d k represents the first dimension of the Q i matrix. Perform multi - head attention extraction on the output of the depth prediction network embedding layer module 7:

[0058] X a =MultiHead(X em ) (5)

[0059] where X a represents the encoded output after multi - head attention.

[0060] Next, decompose the output of the multi - head attention layer to obtain the seasonal and trend features of the input data, effectively predicting the nuclear radiation data. The pattern of the nuclear radiation data itself is very complex and difficult to predict. Divide the data into two parts to better grasp the internal characteristics and laws of the data. Define the module for decomposing the output of the multi - head attention layer as the decomposer. The decomposer performs a padding operation on the input before the feature extraction step, and the padding operation allows the input dimension to be the same as the output dimension. To better utilize the information at both ends of the time - series data, first copy the data at both ends, and then pass through a one - dimensional average pooling layer. Use the average pooling layer to traverse the entire data time - series and find its seasonal and trend features. Convolution kernels of different sizes extract seasonal and trend features of different lengths, better grasping the internal characteristics and laws of the data and improving the prediction accuracy. Obtain the final seasonal and trend terms:

[0061]

[0062] where, Decomposer(·) represents the decomposer operation, and respectively represent the seasonal and trend feature terms of the data in the encoder.

[0063] Then, the convolutional layer is used to learn the seasonal and trend feature terms of the encoder respectively. The output of the convolutional layer passes through the decomposer again to further reveal the temporal correlation of the data:

[0064]

[0065] where and respectively represent the seasonal and trend feature terms of the data in the encoder after feature extraction.

[0066] Finally, the output of the deep prediction network encoder module 8 is obtained by the feedforward layer:

[0067]

[0068] where LayerNorm(·) represents the layer normalization operation, Linear(·) represents the multi-layer perceptron, and X en represents the output of the deep prediction network encoder module 8.

[0069] 3) The deep prediction network decoder module 9, which decodes the output of the deep prediction network encoder module 8 and obtains the final prediction output. The specific process is as follows:

[0070] First, in order to retain the initial features of the original input data, the original input data directly enters the decoder without embedding processing. The first module of the decoder is the decomposer:

[0071]

[0072] where and respectively represent the seasonal and trend feature terms of the data in the decoder, and X de = X t represents the input data of the decoder, that is, the original data.

[0073] Next, the convolutional layer is used to learn the seasonal and trend feature terms of the decoder respectively to obtain the seasonal and trend feature terms in the decoder after feature extraction and

[0074]

[0075] Finally, the output of the deep network encoder module 8 is summed with the seasonal and trend feature terms in the decoder after feature extraction and input to the feedforward layer to obtain the decoder output:

[0076]

[0077] Among them, is the radiation dose rate from the (t + T + 1)-th to the (t + T + H)-th time step predicted by the network, α represents the output proportion factor of the output X of the depth prediction network encoder module en of.

[0078] 4) The depth prediction network training module 10 is used to train the depth prediction network defined in steps 1), 2), and 3), and the training objective is:

[0079]

[0080] Among them, represents the loss function, and the backpropagation is used to train the model during the training process.

[0081] 5) Upload the trained depth prediction network model to the data prediction module 5.

[0082] Specifically, the sensor 3 obtains the radiation dose rate data of nuclear environment monitoring in real time and uploads it to the data prediction module 5. The data prediction module 5 receives the trained depth prediction network model passed in from the supervised training module 4 and uses this model to predict and estimate the real-time monitoring data; finally, the prediction result is displayed on the prediction display module 6.

[0083] The above embodiments are used to explain the present invention, rather than limit the present invention. Any modification and change made to the present invention within the spirit and scope of the claims of the present invention fall within the protection scope of the present invention.

Claims

1. A nuclear power safety radiation dose rate predictor based on attention decomposition mechanism, characterized by: It includes a database, a host computer and a sensor, which are connected in sequence; the database is used to store radiation dose rate data of nuclear power safety history monitoring, and the data stored in the database is used by the host computer; the host computer includes a supervised training module, a data prediction module and a prediction display module which are connected in sequence; the supervised training module of the host computer uses the radiation dose rate data of nuclear power safety history monitoring to train a deep prediction network model, and then uploads the trained deep prediction network model to the data prediction module of the host computer; the sensor acquires nuclear environment monitoring data in real time and uploads it to the data prediction module of the host computer, and the data prediction module predicts and estimates the real-time nuclear environment monitoring data; finally, the prediction result is displayed in the prediction display module of the host computer.

2. A nuclear power safety radiation dose rate predictor based on attention decomposition mechanism as claimed in claim 1, characterized in that: Given the radiation dose rate data of nuclear power safety historical monitoring, referred to as historical monitoring data X t =[x t+1 ,x t+2 ,...,x t+T ], where x t+1 represents the historical monitoring data at the t+1th time step, T represents the length of the time window of the historical monitoring data, Where D represents the dimension of historical monitoring data; is the radiation dose rate from the t+T+1th to the t+T+Hth time step, H represents the length of the prediction data time window; The goal is to learn a function Make 3. A nuclear power safety radiation dose rate predictor based on attention decomposition mechanism as claimed in claim 1, characterized in that: The supervised training module includes a deep prediction network embedding layer module, a deep prediction network encoder module, a deep prediction network decoder module and a deep prediction network training module: 1) A deep prediction network embedding layer module, which performs embedding coding processing on the radiation dose rate data of nuclear power safety history monitoring, including position embedding, tag embedding and time embedding: X em =Position(X in )+Token(X in )+Temporal(X in ) (1) Among them, X in =X t represents the input radiation dose rate data of nuclear power safety history monitoring, Position(·), Token(·) and Temporal(·) represent the position embedding, token embedding and time embedding operations respectively, and X em Represents the output of the deep prediction network embedding layer module; 2) A depth prediction network encoder module, which encodes the output of the depth prediction network embedding layer module to perform feature extraction; the specific process is as follows: First, multi-head attention extraction is performed on the output of the embedding layer module of the deep prediction network: X a =MultiHead(X em ) (5) Where X a Represents the encoded output after multi-head attention; Next, decompose the output of the multi-head attention layer, and define the module that decomposes the output of the multi-head attention layer as a decomposer; the decomposer pads the input before the feature extraction step, and then passes through the one-dimensional average pooling layer, using the average pooling layer to traverse the entire data time series and find its seasonal and trend features. Convolution kernels of different sizes extract seasonal and trend features of different lengths; obtain the final seasonal and trend items: Among them, Decomposer(·) represents the decomposer operation, and Respectively represent the seasonal and trend feature items of the data in the encoder; Then, convolutional layers are used to learn the seasonal and trend feature terms of the encoder respectively; the output of the convolutional layer is passed through the decomposer again to further reveal the temporal correlation of the data: in, and They respectively represent the seasonal and trend feature items of the data in the encoder after feature extraction; Finally, the output of the deep prediction network encoder module is obtained by the feed-forward layer: Among them, LayerNorm(·) represents the layer normalization operation, Linear(·) represents the multi-layer perceptron, and X en Represents the output of the deep prediction network encoder module; 3) A depth prediction network decoder module, which decodes the output of the depth prediction network encoder module and obtains the final prediction output; the specific process is as follows: First, the original data, i.e. the detection data, directly enters the decoder without embedding processing; the first module of the decoder is the decomposer: in, and Represent the seasonal and trend features of the data in the decoder, respectively, X de =X t Represents the input data of the decoder, that is, the original data; Next, the convolutional layer is used to learn the seasonal and trend feature items of the decoder respectively to obtain the seasonal and trend feature items in the decoder after feature extraction. and Finally, the output of the deep network encoder module is summed with the season and trend feature terms in the decoder after feature extraction and input into the feed-forward layer to obtain the decoder output: in, is the radiation dose rate predicted by the network from time step t+T+1 to time step t+T+H, α represents the output X of the deep prediction network encoder module en Output proportion factor of 4) A depth prediction network training module, which is used to train a depth prediction network model, and the training objectives are: in, Represents the loss function, and the back-propagation training model is used during training; 5) Upload the trained deep prediction network model to the data prediction module.

4. A nuclear power safety radiation dose rate predictor based on attention decomposition mechanism as claimed in claim 3, characterized in that: Multi-head attention is expressed as: MultiHead(Q,K,V)=Concat(h1,h2,…,h n )W (2) h i =Attention(Q i ,K i ,V i ) (3) where MultiHead(·) represents a multi-head attention operation, Concat(·) represents a concatenation operation, Attention(·) represents an attention operation, SoftMax(·) represents a normalization operation on the attention score, Q, K, and V represent query, key, and value matrices, and h i represents the i-th attention head, 1≤i≤n, n represents the total number of attention heads, W represents the mapping matrix, Q i , K i and V i represents the i-th query, key and value matrix, d k Indicates Q i The first dimension of the matrix.

5. A nuclear power safety radiation dose rate predictor based on attention decomposition mechanism as claimed in claim 1, characterized in that: The sensor acquires radiation dose rate data of nuclear environment monitoring in real time and uploads it to the data prediction module. The data prediction module receives the trained deep prediction network model transmitted from the supervised training module, and uses the model to predict and estimate the real-time monitoring data; finally, the prediction result is displayed in the prediction display module.

Images