Environmental dose dynamic prediction method and system based on multi-modal edge fusion

Abstract

The present application belongs to the technical field of radiation monitoring and artificial intelligence, comprising an environmental dose dynamic prediction method and system based on multi-modal edge fusion, wherein the method comprises: acquiring multi-modal data of a radiation monitoring area and performing time synchronization; encoding the multi-modal data according to the data structure to obtain encoded data; performing multi-modal fusion on the encoded data using a multi-head attention mechanism to obtain fusion data; predicting the fusion data using a lightweight time sequence convolution network to obtain a dose rate prediction value at a future short time; determining a confidence interval using a quantile regression method according to the dose rate prediction value; and performing adaptive early warning on the dose rate prediction value at the future short time of the radiation monitoring area according to the dose rate prediction value, the confidence interval and the early warning threshold, thereby realizing scene perception and adaptability of the environmental dose and improving accurate judgment under complex working conditions.

Description

Technical Field

[0001] This invention relates to the fields of radiation monitoring and artificial intelligence technology, and in particular to a method and system for dynamic prediction of environmental dose based on multimodal edge fusion. Background Technology

[0002] Electron linear accelerators, due to their advantages such as concentrated and stable beams, strong penetration capabilities, and reliable safety, have been widely used in fields such as medical and health care, industrial irradiation, agricultural breeding, materials modification, and national security. In irradiation environments, accelerators generate high-energy X-rays during operation, and when the electron beam energy exceeds 10 MeV, neutron radiation is also produced through photonuclear reactions. Negligence in radiation management can pose external radiation hazards to workers and the public.

[0003] The following defects exist in environmental dose monitoring: (1) Poor adaptability to working conditions. Currently, detection relies on fixed or portable single detectors, resulting in a single data dimension. At the same time, fixed threshold settings are generally used. Judging "dose rate exceeding the threshold" can only adapt to static or stable environments, which is not adaptable to complex working conditions and has a high false alarm rate. (2) Lack of predictive ability. It can only achieve post-event alarm and cannot provide early warning before abnormal exposure occurs. (3) Data silos and weak collaborative perception capabilities. Although the production environment is equipped with safety facilities such as radiation monitoring instruments, door interlocks, and video surveillance, dose rate monitoring only focuses on the time series changes of γ / neutron count rate, video surveillance is only used for post-event traceability of personnel entry and exit, and equipment status signals (door magnetic switches, beam output signals) are only used as safety interlock trigger conditions. There is a lack of real-time collaboration and correlation between the three types of data streams, forming data silos and making it impossible to build a complete safety collaborative perception scenario. The above problems result in radiation protection management always being in a passive response state, which is difficult to meet the needs of radiation processing enterprises for refined safety management in the process of intelligent transformation.

[0004] To address the aforementioned shortcomings, existing technologies suffer from the following deficiencies: Firstly, existing solutions lack sufficient multimodal data fusion and a cross-modal attention mechanism, making adaptive adjustments impossible based on scene changes. Secondly, existing solutions primarily focus on comparing real-time monitoring values ​​with thresholds for further interlocking alarms. The system only triggers alarms and interlocking actions when the current measured dose rate exceeds a set threshold, failing to output dose rate predictions for future time periods. This constitutes a reactive response mechanism lacking predictive capability; by the time an alarm is triggered, personnel may have already received a certain dose, failing to achieve preventative measures. Thirdly, existing solutions do not explicitly complete multimodal feature encoding, attention fusion, and dynamic prediction at the edge, resulting in limited real-time performance and bandwidth, thus failing to meet the requirements for second-level early warning. Summary of the Invention

[0005] Aimed at at least in solving one of the technical problems existing in the prior art, the present invention provides a method and system for dynamic prediction of environmental dose based on multimodal edge fusion.

[0006] One aspect of the present invention provides a method for dynamic prediction of environmental dose based on multimodal edge fusion, comprising: Acquire multimodal data of the radiation monitoring area and synchronize it in time. The multimodal data includes radiation monitoring data, equipment status data, and personnel video data. For multimodal data, encoding is performed according to the data structure to obtain encoded data; The encoded data is fused using a multi-head attention mechanism to obtain fused data; A lightweight temporal convolutional network is used to predict the dose rate in the fused data to obtain the dose rate prediction value in the near future. Confidence intervals were determined using quantile regression based on the dose rate predictions. Adaptive early warning is provided for the dose rate prediction values ​​in the radiation monitoring area in the short term based on the dose rate prediction values, confidence intervals, and early warning thresholds.

[0007] According to the aforementioned environmental dose dynamic prediction method based on multimodal edge fusion, acquiring multimodal data of the radiation monitoring area and performing time synchronization includes: Obtain readings from the high-voltage ionization chamber and the neutron Rem instrument, which sample at a preset sampling frequency, and obtain radiation monitoring data from the readings in a time series: ; in, Represents a radiation monitoring data vector. For high-voltage ionization chamber Number of detectors This represents the number of neutron detectors. Indicates in Readings from the time detector; Indicates transpose; Obtain key equipment status data during accelerator operation to obtain equipment status data, where the equipment status data consists of discrete categorical variables. for: ; in, Indicates the accelerator at The operating status at any given time: 1 indicates the output state, and 0 indicates the standby or stop state. Indicates that the protective door magnet is in The on / off state at any given time, 1 indicates the off state and 0 indicates the on state; Acquire video data collected by cameras located in the radiation monitoring area, and use a lightweight target detection network to detect people using the video data: The presence status of people in the video data is detected frame by frame, and the presence status is as follows: ; Then, a sliding window de-jitter mechanism is used on the video data to re-determine the personnel detection status: ; in, This is the time index variable within the window; The length of the sliding window. The sensitivity threshold, The trigger threshold for the number of frames in which the number of people needs to be detected within the window; Based on the personnel detection status, personnel video data is generated, where the personnel video data is a structured feature vector. ; If the sampling frequencies of multimodal data are the same, time alignment is performed using device status data; If the sampling frequencies of multimodal data are different, time alignment is performed using nearest neighbor interpolation.

[0008] According to the aforementioned environmental dose dynamic prediction method based on multimodal edge fusion, the multimodal data is encoded according to a data structure to obtain encoded data, including: If the multimodal data is radiation monitoring data, the readings of the radiation detectors in historical time periods are obtained, and the historical readings are concatenated into a time window sequence. Temporal feature extraction, fully connected layer mapping, and ReLU activation are then performed using dilated causal convolution to obtain the radiation features: ; ; in, For a moment radiative eigenvectors; For dilated causal convolution operations, directly on Historical radiation data sequence in seconds Perform calculations; Slice the input sequence from arrive Dose rate at time; This is the weight matrix of the fully connected layer; Here, represents the bias vector of the fully connected layer; ReLU is the activation function. The length of the historical period; If the multimodal data is equipment status data, the discrete categorical variables of the equipment status data are mapped to a continuous vector space using an embedding method to obtain the equipment status features as follows: ; in, Let E be the device state feature vector, and E be the embedding matrix; For state combination index functions; The status value of the device; If the multimodal data consists of personnel video data, an LSTM network gating mechanism is used to process the sparse event sequences of the personnel video data. This includes inputting the structured feature vector of the personnel video data at each time step and discarding it using a forget gate. ; ; in, For a moment t The output of the forget gate; Use the Sigmoid activation function; Here is the weight matrix for the forget gate; To offset the forget gate; To splice the hidden state from the previous moment and current input ; The Sigmoid activation function has an output range of... ; Structured feature vectors for personnel video data; For personnel status monitoring; The current input is processed through the input gate. Update: ; ; The input gate includes the gate control signal of the Sigmoid layer. Candidate cell state of Tanh layer ,in For input gates; For the Sigmoid function; and Represents the weight matrix; The content to generate new information; tanh is the tangent activation function, with an output range of (-1, 1); Combining the outputs of the forget gate and the input gate, the updated cell state is obtained as follows: ; in, For a moment Cellular memory state; ⊙ represents element-wise multiplication; It is the output of the forget gate; This indicates that old information has been forgotten; This table shows the addition of new information; From the current cellular memory state, the hidden state is obtained, and the characteristics of events involving personnel are as follows: ; ; ; in, For output signal; For a moment The hidden state output; The event characteristics of the personnel are represented by the hidden state of the last time step.

[0009] According to the aforementioned environmental dose dynamic prediction method based on multimodal edge fusion, the encoded data is fused using a multi-head attention mechanism to obtain fused data, including: Project the encoded data into a one-dimensional space. The projection features are obtained, including the projected radiation features. Characteristics of events occurring after projection Device status characteristics after projection The formula for calculating the projection feature is: ; ; ; in, , These are all feature projection matrices, used to map 64-dimensional or 16-dimensional data to 128-dimensional data. For layer normalization; Attention points for obtaining projected features: ; in, This is the feature vector for the current query. This is a key matrix containing the content to be queried. This is the value matrix of the content to be queried. This is the attention weighting coefficient. To query the similarity matrix with the key, This is the scaling factor; Radiation monitoring - personnel attention Capturing the correlation between radiation dose rate and human activity; focusing on radiation monitoring and equipment status. Capturing the correlation between radiation dose rate and equipment status; using self-attention heads for radiation monitoring The temporal dependence of the radiation dose rate sequence under normal operating conditions is captured, where: ; ; ; ; ; ; Radiation monitoring - personnel attention Radiation monitoring - equipment status attention and radiation monitoring self-attention head Perform weighted fusion, and then obtain the fusion features by connecting the weighted fusion results through residual connections: ; ; in, For the weighted fusion result, As a feature of fusion, The fusion coefficient between attentions These are the weighting coefficients within the attention span.

[0010] According to the aforementioned environmental dose dynamic prediction method based on multimodal edge fusion, the fused data is predicted using a lightweight temporal convolutional network to obtain the dose rate prediction value for the next short time, including: Before inputting the fused data into a lightweight temporal convolutional network, the real-time monitored data sequence and the fused feature sequence are acquired and concatenated along the feature dimension to obtain the joint input. for: ; in, The dose rate monitoring value at the current moment (real time) ; For a moment The fusion characteristics; Dilated convolution is performed using a lightweight temporal convolutional network, with the following formula: ; in, For the input layer, For the first Layer at time The output, The kernel size is [size]. For the first The expansion rate of the layer, and These are learnable parameters; Perform residual connections at each layer of dilated convolution: ; go through After a layer of dilated convolution, the result is mapped to the prediction output through a fully connected layer, yielding the prediction result for the next short time step. For each time step prediction, extract the output of the last layer at the corresponding time step: ; in, The total number of time steps in the near future. For the short term of the future Dose rate predictions for each time step.

[0011] According to the aforementioned environmental dose dynamic prediction method based on multimodal edge fusion, the confidence interval is determined using a quantile regression method based on the dose rate prediction value, including: The same fused data is simultaneously fed into three parallel lightweight temporal convolutional networks. , get output , , The three equal values ​​are the lower limit of the confidence interval, the median of the confidence interval, and the upper limit of the confidence interval, as shown in the following formula: ; in, For the next short time Quantile predictions of dose rate at each time step; For quantiles; For the corresponding quantiles Trained independent lightweight temporal convolutional prediction network; for Historical dose rate sequence at each time step; The output is the fused feature sequence; Should Unique parameters of quantile networks.

[0012] According to the aforementioned environmental dose dynamic prediction method based on multimodal edge fusion, adaptive early warning is performed on the predicted dose rate of the radiation monitoring area in the short term based on the dose rate prediction value, confidence interval, and early warning threshold, including: The triggering conditions for the early warning mechanism are determined based on the dose rate prediction and confidence interval. These triggering conditions include attention triggering conditions, early warning triggering conditions, and alarm triggering conditions. The attention triggering conditions are as follows: ; The trigger condition indicates the predicted value of the median. Exceeding the warning threshold However, the lower limit of the confidence interval Not exceeding ; The warning trigger conditions are: ; The warning trigger condition represents the lower limit of the confidence interval. Exceeding the warning threshold ; Alarm trigger conditions: ; The alarm trigger condition indicates that the lower limit of the confidence interval exceeds the alarm threshold. , This represents two different thresholds; When the predicted dose rate of the radiation monitoring area at a future time meets the triggering conditions of the early warning mechanism, the corresponding early warning action will be executed.

[0013] Another aspect of the present invention provides an environmental dose dynamic prediction system based on multimodal edge fusion, comprising: The first module is used to acquire multimodal data of the radiation monitoring area and synchronize it in time. The multimodal data includes radiation monitoring data, equipment status data and personnel video data. The second module is used to encode multimodal data according to the data structure to obtain encoded data; The third module is used to perform multimodal fusion on the encoded data using a multi-head attention mechanism to obtain fused data; The fourth module is used to perform prediction on the fused data using a lightweight temporal convolutional network to obtain the dose rate prediction value in the near future. The fifth module is used to determine the confidence interval based on the dose rate prediction using the quantile regression method; The sixth module is used to provide adaptive early warning of the dose rate prediction value for the radiation monitoring area in the short term, based on the dose rate prediction value, confidence interval, and early warning threshold.

[0014] The beneficial effects of this invention are as follows: It encodes features of comprehensive perception data such as radiation dose rate, equipment operating conditions, and personnel safety videos at edge nodes, embeds causal relationships of complex operating conditions, and performs multimodal data fusion, enabling the system to perceive the scene of dose changes; it constructs a prediction model based on a lightweight temporal convolutional network and deploys it at the edge, performing dose prediction in a short time based on multimodal fusion feature data and real-time monitored dose data, predicting dose rate change trends in advance and achieving proactive risk prevention; and it utilizes a multi-head attention mechanism to construct causal relationships of complex operating conditions such as equipment status-dose response and personnel presence-dose response, enhancing the adaptability of prediction and achieving accurate judgment of complex operating conditions. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the environmental dose dynamic prediction method based on multimodal edge fusion according to an embodiment of the present invention.

[0016] Figure 2 This is a schematic diagram of an environmental dose dynamic prediction system based on multimodal edge fusion according to an embodiment of the present invention. Detailed Implementation

[0017] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings. Throughout the description, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions. In the following description, suffixes such as "module," "part," or "unit" used to denote elements are used only for the purpose of illustrative purposes and have no specific meaning in themselves. Therefore, "module," "part," or "unit" can be used interchangeably. Terms such as "first," "second," etc., are used only to distinguish technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the sequential relationship of the indicated technical features. In the following description, the consecutive reference numerals for method steps are for ease of review and understanding. Adjusting the implementation order of steps, in conjunction with the overall technical solution of the present invention and the logical relationship between the various steps, will not affect the technical effect achieved by the technical solution of the present invention. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0018] refer to Figure 1 , Figure 1 This is a schematic flowchart of the environmental dose dynamic prediction method based on multimodal edge fusion according to an embodiment of the present invention, which includes, but is not limited to, steps S100~S600: S100 acquires multimodal data of the radiation monitoring area and performs time synchronization, including radiation monitoring data, equipment status data, and personnel video data.

[0019] In some embodiments, to fully cover the mixed radiation field, a dual-detector configuration is deployed at key locations such as protective doors: a high-voltage ionization chamber and a neutron Rem meter, used for gamma dose rate monitoring and neutron dose equivalent monitoring, respectively; the sampling frequency is uniformly set to 5Hz, that is, one data point is collected every 5 seconds, and the collected data is a time series, as follows: Obtain readings from the high-voltage ionization chamber and the neutron Rem instrument, which sample at a preset sampling frequency, and obtain radiation monitoring data from the readings in a time series: ; in, Represents a radiation monitoring data vector. For high-voltage ionization chamber Number of detectors This represents the number of neutron detectors. Indicates in Readings from the time detector; This indicates transpose.

[0020] In some embodiments, equipment status data is primarily used to identify complex operating conditions and reduce the false alarm rate of subsequent warnings. Currently, in the safety interlocking mechanism of the working environment, the access control is closed while the accelerator is running. When the accelerator stops, the access control can be opened; therefore, both types of data need to be collected simultaneously for auxiliary identification. When collecting operating status data such as accelerator running status and access control status through the PLC interface, the equipment status vector is a discrete categorical variable, as follows: Obtain key equipment status data during accelerator operation to obtain equipment status data, where the equipment status data consists of discrete categorical variables. for: ; in, Indicates the accelerator at The operating status at any given time: 1 indicates the output state, and 0 indicates the standby or stop state. Indicates that the protective door magnet is in The on / off state at any given time, where 1 indicates the off state and 0 indicates the on state.

[0021] In some embodiments, network cameras are deployed in key detection areas such as protective doors to acquire video data of personnel presence. This primarily involves obtaining information about personnel's presence within the detection area. The system uses detection and analysis to prevent personnel from accidentally entering or illegally lingering outside the detection area, triggering a real-time security alarm in case of such incidents. A lightweight target detection network (using YOLOv8-nano) is deployed at the edge to perform second-level personnel detection (primarily determining whether personnel are present within the detection area), ultimately outputting a feature vector, as detailed below: Acquire video data collected by cameras located in the radiation monitoring area, and use a lightweight target detection network to detect people using the video data: The presence status of people in the video data is detected frame by frame, and the presence status is as follows: ; Then, a sliding window de-jitter mechanism is used on the video data to re-determine the personnel detection status: ; in, This is the time index variable within the window; The length of the sliding window. The sensitivity threshold, The trigger threshold for the number of frames in which the number of people needs to be detected within the window; Based on the personnel detection status, personnel video data is generated, where the personnel video data is a structured feature vector. .

[0022] In some embodiments, time alignment includes: if the sampling frequencies of the multimodal data are the same, time alignment is performed using device state data; if the sampling frequencies of the multimodal data are different, time alignment is performed using nearest neighbor interpolation.

[0023] For example, time alignment of multimodal data facilitates fusion processing. For data with the same sampling frequency, time synchronization of the acquisition devices is set; for data with different sampling frequencies, nearest neighbor interpolation is used to align them to a unified time grid. The nearest neighbor measurement value is taken for each modality at any given time.

[0024] S200 encodes multimodal data according to the data structure to obtain encoded data.

[0025] In some embodiments, the multimodal data structures vary, therefore they need to be encoded separately and then mapped to a unified feature space, as follows: It should be noted that dose rate sequences in radiation monitoring data exhibit a certain long-term dependence, particularly historical dose rates, which have a sustained impact on future predictions. These data also show slow trends or sudden spikes across multiple time scales. Dilated causal convolution can effectively model these characteristics and obtain the time-series data. radiative eigenvectors .

[0026] Specifically, if the multimodal data is radiation monitoring data, the readings of the radiation detectors in historical time periods are obtained, and the historical readings are concatenated into a time window sequence. Temporal feature extraction, fully connected layer mapping, and ReLU activation are then performed through dilated causal convolution to obtain the radiation features: ; ; in, For a moment radiative eigenvectors; For dilated causal convolution operations, directly on Historical radiation data sequence in seconds Perform calculations; Slice the input sequence from arrive Dose rate at time; This is the weight matrix of the fully connected layer; Here, represents the bias vector of the fully connected layer; ReLU is the activation function. The length of the historical period.

[0027] If the multimodal data is equipment status data, the discrete categorical variables of the equipment status data are mapped to a continuous vector space using an embedding method to obtain the equipment status features as follows: ; in, Let E be the device state feature vector, and E be the embedding matrix. For state combination index functions; The device state value; wherein, the embedding method in this embodiment makes discrete variables continuous, and the device state combination has a total of The possible values ​​are given by the Embedding matrix. .

[0028] In some embodiments, if the multimodal data is personnel video data, the sparse event sequence of the personnel video data is processed using the gating mechanism of an LSTM network. This includes inputting a structured feature vector of the personnel video data at each time step and discarding it through a forget gate (which determines which historical data to discard). ; ; That is, when a person appears in the testing area Most of the time The video event feature sequence is a binary sequence. This scheme uses the gating mechanism of an LSTM network to obtain this type of sparse event sequence. The input video event feature vector of LSTM at each time step ; For a moment t The output of the forget gate; Use the Sigmoid activation function; Here is the weight matrix for the forget gate; To offset the forget gate; To splice the hidden state from the previous moment and current input ; The Sigmoid activation function has an output range of... ; Structured feature vectors for personnel video data; For personnel status monitoring.

[0029] Further through the input gate, it is controlled by the gate signal of the Sigmoid layer. Candidate cell state of Tanh layer It consists of two parts, which are ultimately combined to form the updated state. : ; ; in, Candidate cell state, This is an input gate (controlling the proportion of new information entering); For the Sigmoid function; and Represents the weight matrix; The content to generate new information; tanh is the tangent activation function, with an output range of (-1, 1); Combining the outputs of the forget gate and the input gate, the updated cell state is obtained as follows: ; in, For a moment t Cellular memory state; ⊙ represents element-wise multiplication; It is the output of the forget gate; This indicates that old information has been forgotten; This table shows the addition of new information; From the current cellular memory state, the hidden state is obtained, and the characteristics of events involving personnel are as follows: ; ; ; in, For output signal; For a moment The hidden state output; The event characteristics of the personnel are represented by the hidden state of the last time step.

[0030] S300, the encoded data is fused using a multi-head attention mechanism to obtain fused data.

[0031] Understandably, in complex work scenarios, relying on single-dimensional radiation data for early warning cannot explain "why the radiation level has increased," nor can it distinguish between normal radiation emission and leakage. Assessing personnel safety in critical areas from a single perspective fails to quantify radiation risk or determine the current dose level. Using equipment in isolation cannot assess its actual impact on personnel. Therefore, it is necessary to integrate data from radiation dose monitoring, personnel presence, and equipment status to construct a causal correlation mechanism to adapt to risk assessments under varying work conditions. To reduce bandwidth pressure, network transmission latency, and personnel privacy risks, multimodal fusion processing should be performed at the edge to replace the real-time and bandwidth bottlenecks of centralized cloud processing, thus meeting real-time processing requirements.

[0032] In some embodiments, the multi-head cross-attention mechanism queries video features and state features from the perspective of radial features, as follows: Project the encoded data into a one-dimensional space. The projection features are obtained, including the projected radiation features. Characteristics of events occurring after projection Device status characteristics after projection The formula for calculating the projection feature is: ; ; ; in, , These are all feature projection matrices, used to map 64-dimensional or 16-dimensional data to 128-dimensional data. For layer normalization; Attention points for obtaining projected features: ; in, This is the feature vector for the current query. This is a key matrix containing the content to be queried. This is the value matrix of the content to be queried. This is the attention weighting coefficient. To query the similarity matrix with the key, This is the scaling factor; Radiation monitoring - personnel attention Capturing the correlation between radiation dose rate and human activity; focusing on radiation monitoring and equipment status. Capturing the correlation between radiation dose rate and equipment status; using self-attention heads for radiation monitoring The temporal dependence of the radiation dose rate sequence under normal operating conditions is captured, where: ; ; ; ; ; ; Radiation monitoring - personnel attention Radiation monitoring - equipment status attention and radiation monitoring self-attention head Perform weighted fusion, and then obtain the fusion features by connecting the weighted fusion results through residual connections: ; ; in, For the weighted fusion result, As a feature of fusion, The fusion coefficient between attentions These are the weighting coefficients within the attention span.

[0033] It is understood that the embodiments of the present invention employ dynamic attention weights to achieve adaptation to the working environment. The weight coefficients within multi-head attention... It changes dynamically with the input data, through Real-time calculation, combined with the fusion coefficient between attention. This is used to balance the contribution ratios of the three association modes. and These factors collectively determine the model's focus under different operating conditions.

[0034] As personnel approach the scene, when they appear in the area of ​​the protective door... The attention weight is significantly increased, making the model more sensitive to human behavior and allowing for early intervention against risks; in equipment startup scenarios, when the accelerator beam exits or the gate magnetic state changes abruptly... The attention weight will increase instantaneously, and rapid response to changes in device status may lead to radiation risks; in normal beam output scenarios, when the accelerator is in a stable beam output state and there is no personnel activity, Attention weights dominate, and the model primarily relies on historical radiation data. This correlation pattern can automatically adjust the focus based on different operating conditions.

[0035] S400, a lightweight temporal convolutional network is used to predict the fused data to obtain the dose rate prediction value for the next short time.

[0036] To meet the high-speed computing requirements at the edge, a lightweight temporal convolutional network (TCN-Lite) is used for short-term dose rate prediction. TCN-Lite features a small number of parameters, fast computation speed, and low inference latency, making it suitable for deployment in resource-constrained edge gateways. The core of TCN-Lite consists of several layers of causal dilated convolutions stacked together. The input sequence is the past... Fusion features at each time step and real-time monitored dose rate sequence After dilated convolution and residual connection, the future output is obtained. Dose rate prediction at each time step .

[0037] In some embodiments, before inputting the fused data into a lightweight temporal convolutional network, the real-time monitored data sequence and the fused feature sequence are acquired and concatenated along the feature dimension to obtain the joint input. for: ; in, The dose rate monitoring value at the current moment (real time) ; For a moment The fusion characteristics; Dilated convolution is performed using a lightweight temporal convolutional network, with the following formula: ; in, For the input layer, For the first Layer at time The output, The kernel size is [size]. For the first The expansion rate of the layer, and These are learnable parameters; Perform residual connections at each layer of dilated convolution: ; go through After a layer of dilated convolution, the result is mapped to the prediction output through a fully connected layer, yielding the prediction result for the next short time step. For each time step prediction, extract the output of the last layer at the corresponding time step: ; in, The total number of time steps in the near future. For the short term of the future Dose rate predictions for each time step.

[0038] S500, based on the dose rate prediction, uses quantile regression to determine the confidence interval.

[0039] In some embodiments, a quantile regression method is used to predict multiple quantiles simultaneously. For the quantile... Step prediction, output respectively The predicted values ​​for the three quantiles include: The same fused data is simultaneously fed into three parallel lightweight temporal convolutional networks. , get output Alarm threshold lower limit Alarm threshold (number of Chinese characters) The upper limit of the alarm threshold has three equal values: the lower limit of the confidence interval, the median of the confidence interval, and the upper limit of the confidence interval, as shown in the following formula: ; in, For the next short time Quantile predictions of dose rate at each time step; For quantiles; For the corresponding quantiles Trained independent lightweight temporal convolutional prediction network; Historical dose rate sequence; The output is the fused feature sequence; Should Unique parameters of quantile networks.

[0040] S600 provides adaptive early warnings based on the dose rate prediction, confidence interval, and warning threshold for the short-term dose rate prediction of the radiation monitoring area.

[0041] In some embodiments, the triggering conditions for the early warning mechanism are determined based on the dose rate prediction value and the confidence interval, wherein the triggering conditions include attention triggering conditions, early warning triggering conditions, and alarm triggering conditions, wherein the attention triggering condition is: ; Focus on trigger conditions representing the median The predicted value exceeds the warning threshold. However, the lower limit of the confidence interval Not exceeding; The warning trigger conditions are: ; The warning trigger condition represents the lower limit of the confidence interval. Exceeding the warning threshold ; Alarm trigger conditions: ; Alarm triggering conditions represent the lower limit of the confidence interval. Exceeding alarm threshold , This represents two different thresholds; When the predicted dose rate of the radiation monitoring area in the near future meets the triggering conditions of the early warning mechanism, the corresponding early warning action is executed.

[0042] For example, where 1.5 μSv / h 2.5 μSv / h.

[0043] Figure 2This is a schematic diagram of an environmental dose dynamic prediction system based on multimodal edge fusion according to an embodiment of the present invention. The system includes a first module 210, a second module 220, a third module 230, a fourth module 240, a fifth module 250, and a sixth module 260.

[0044] The system comprises six modules: a first module for acquiring and synchronizing multimodal data of the radiation monitoring area, including radiation monitoring data, equipment status data, and personnel video data; a second module for encoding the multimodal data according to its structure; a third module for fusing the encoded data using a multi-head attention mechanism; a fourth module for predicting the dose rate of the fused data using a lightweight temporal convolutional network; a fifth module for determining the confidence interval based on the dose rate prediction using quantile regression; and a sixth module for adaptively issuing early warnings based on the dose rate prediction, confidence interval, and warning threshold for the short-term dose rate prediction of the radiation monitoring area.

[0045] For example, with the cooperation of the first, second, third, fourth, fifth, and sixth modules in the system, the embodiment device can implement any of the aforementioned methods for dynamic prediction of environmental dose based on multimodal edge fusion. This involves acquiring multimodal data of the radiation monitoring area and synchronizing it in time. The multimodal data includes radiation monitoring data, equipment status data, and personnel video data. The multimodal data is encoded according to a data structure to obtain encoded data. The encoded data is then fused using a multi-head attention mechanism to obtain fused data. The fused data is then predicted using a lightweight temporal convolutional network to obtain a predicted dose rate value for the short future moments. Based on the predicted dose rate value, a quantile regression method is used to determine the confidence interval. Finally, an adaptive warning is issued for the predicted dose rate value of the radiation monitoring area for the short future moments based on the predicted dose rate value, the confidence interval, and the warning threshold. The beneficial effects of this invention are as follows: It encodes features of comprehensive perception data such as radiation dose rate, equipment operating conditions, and personnel safety videos at edge nodes, embeds causal relationships of complex operating conditions, and performs multimodal data fusion, enabling the system to perceive the scene of dose changes; it constructs a prediction model based on a lightweight temporal convolutional network and deploys it at the edge, performing dose prediction in a short time based on multimodal fusion feature data and real-time monitored dose data, predicting dose rate change trends in advance and achieving proactive risk prevention; and it utilizes a multi-head attention mechanism to construct causal relationships of complex operating conditions such as equipment status-dose response and personnel presence-dose response, enhancing the adaptability of prediction and achieving accurate judgment of complex operating conditions.

[0046] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented in the embodiments of this invention. Alternative embodiments are contemplated, in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0047] This invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned environmental dose dynamic prediction method based on multimodal edge fusion.

[0048] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the described functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, considering the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed in the embodiments of the invention, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0049] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0050] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can include, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0051] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0052] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0053] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0054] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0055] The above is a detailed description of the preferred embodiments of the present invention, but the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention, and these equivalent modifications or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A method for dynamic prediction of environmental dose based on multimodal edge fusion, characterized in that, include: Acquire multimodal data of the radiation monitoring area and synchronize it in time. The multimodal data includes radiation monitoring data, equipment status data, and personnel video data. For multimodal data, encoding is performed according to the data structure to obtain encoded data; The encoded data is fused using a multi-head attention mechanism to obtain fused data; A lightweight temporal convolutional network is used to predict the dose rate in the fused data to obtain the dose rate prediction value in the near future. Confidence intervals were determined using quantile regression based on the dose rate predictions. Adaptive early warning is provided for the dose rate prediction values ​​in the radiation monitoring area in the short term based on the dose rate prediction values, confidence intervals and early warning thresholds. The feature is that the encoding of multimodal data according to a data structure to obtain encoded data includes: If the multimodal data is radiation monitoring data, the readings of the radiation detectors in historical time periods are obtained, and the historical readings are concatenated into a time window sequence. Temporal feature extraction, fully connected layer mapping, and ReLU activation are then performed using dilated causal convolution to obtain the radiation features: in, It is a radiation eigenvector; For dilated causal convolution operations, directly on Historical radiation monitoring data sequence in seconds Perform calculations; This is the weight matrix of the fully connected layer; Here, represents the bias vector of the fully connected layer; ReLU is the activation function. The length of the historical period; express Real-time radiation monitoring data; If the multimodal data is equipment status data, the discrete categorical variables of the equipment status data are mapped to a continuous vector space using an embedding method to obtain the equipment status features as follows: in, Let E be the device state feature vector, and E be the embedding matrix; For state combination index functions; The operating status of the accelerator, To protect the on / off state of the door magnetic sensor; If the multimodal data consists of personnel video data, an LSTM network gating mechanism is used to process the sparse event sequences of the personnel video data. This includes inputting the structured feature vector of the personnel video data at each time step and discarding it using a forget gate. in, For a moment t The output of the forget gate; Use the Sigmoid activation function; Here is the weight matrix for the forget gate; To offset the forget gate; To splice the hidden state from the previous moment and current input ; The Sigmoid activation function has an output range of... ; Structured feature vectors for personnel video data; For personnel status monitoring; The current input is processed through the input gate. Update: The input gate includes the gate control signal of the Sigmoid layer. Candidate cell state of Tanh layer ,in For input gates; For the Sigmoid function; and Represents the weight matrix; The content to generate new information; tanh is the tangent activation function, with an output range of (-1, 1); Combining the outputs of the forget gate and the input gate, the updated cell state is obtained as follows: in, For a moment t Cellular memory state; ⊙ represents element-wise multiplication; It is the output of the forget gate; This indicates that old information has been forgotten; This table shows the addition of new information; From the current cellular memory state, the hidden state is obtained, and the characteristics of events involving personnel are as follows: in, For output signal; For a moment The hidden state output; The event characteristics of the personnel are represented by the hidden state of the last time step. The multi-head attention mechanism is used to perform multimodal fusion on the encoded data to obtain fused data, including: Project the encoded data into a one-dimensional space. The projection features are obtained, including the projected radiation features. Characteristics of events occurring after projection Device status characteristics after projection The formula for calculating the projection feature is: in, , These are all feature projection matrices, used to map 64-dimensional or 16-dimensional data to 128-dimensional data. For layer normalization; Attention points for obtaining projected features: in, This is the feature vector for the current query. This is a key matrix containing the content to be queried. This is the value matrix of the content to be queried. This is the attention weighting coefficient. To query the similarity matrix with the key, This is the scaling factor; Radiation monitoring - personnel attention Capturing the correlation between radiation dose rate and human activity; focusing on radiation monitoring and equipment status. Capturing the correlation between radiation dose rate and equipment status; using self-attention heads for radiation monitoring The temporal dependence of the radiation dose rate sequence under normal operating conditions is captured, where: Radiation monitoring - personnel attention Radiation monitoring - equipment status attention and radiation monitoring self-attention head Perform weighted fusion, and then obtain the fusion features by connecting the weighted fusion results through residual connections: in, For the weighted fusion result, As a feature of fusion, The fusion coefficient between attentions These are the weighting coefficients within the attention span.

2. The environmental dose dynamic prediction method based on multimodal edge fusion according to claim 1, characterized in that, The acquisition and time synchronization of multimodal data of the radiation monitoring area includes: Obtain readings from the high-voltage ionization chamber and the neutron Rem instrument, which sample at a preset sampling frequency, and obtain radiation monitoring data from the readings in a time series: in, express Real-time radiation monitoring data, For high-voltage ionization chamber Number of detectors This represents the number of neutron detectors. Indicates in Readings from the time detector; Indicates transpose; Obtain key equipment status data during accelerator operation to obtain equipment status data, where the equipment status data consists of discrete categorical variables. for: in, Indicates the accelerator at The operating status at any given time: 1 indicates the output state, and 0 indicates the standby or stop state. Indicates that the protective door magnet is in The on / off state at any given time, 1 indicates the off state and 0 indicates the on state; Acquire video data collected by cameras positioned in the radiation monitoring area, and use a lightweight target detection network to detect personnel using the video data: The presence status of people in the video data is detected frame by frame, and the presence status is as follows: Then, a sliding window de-jitter mechanism is used on the video data to re-determine the personnel detection status: in, This is the time index variable within the window; The length of the sliding window. The sensitivity threshold, The trigger threshold for the number of frames in which the number of people needs to be detected within the window; Based on the personnel detection status, personnel video data is generated, where the personnel video data is a structured feature vector. ; If the sampling frequencies of multimodal data are the same, time alignment is performed using device status data; If the sampling frequencies of multimodal data are different, time alignment is performed using nearest neighbor interpolation.

3. The environmental dose dynamic prediction method based on multimodal edge fusion according to claim 1, characterized in that, The step of using a lightweight temporal convolutional network to predict the fused data to obtain predicted dose rates for short future moments includes: Before inputting the fused data into a lightweight temporal convolutional network, the real-time monitored data sequence and the fused feature sequence are acquired and concatenated along the feature dimension to obtain the joint input. for: in, for Real-time radiation monitoring data ; for The fusion characteristics of moments; Dilated convolution is performed using a lightweight temporal convolutional network, with the following formula: in, For the input layer, For the first Layer at time The output, The kernel size is [size]. For the first The expansion rate of the layer, and These are learnable parameters; Perform residual connections at each layer of dilated convolution: go through After a layer of dilated convolution, the result is mapped to the prediction output through a fully connected layer, yielding the prediction result for the next short time step. For each time step prediction, extract the output of the last layer at the corresponding time step: in, The total number of time steps in the near future. For the short term of the future Dose rate predictions at each time step and These are learnable parameters.

4. The environmental dose dynamic prediction method based on multimodal edge fusion according to claim 3, characterized in that, The step of determining the confidence interval using quantile regression based on the dose rate prediction includes: The same fused data is simultaneously fed into three parallel lightweight temporal convolutional networks. , get output , , The three equal values ​​are the lower limit of the confidence interval, the median of the confidence interval, and the upper limit of the confidence interval, as shown in the following formula: in, For the next short time Quantile predictions of dose rate at each time step; For quantiles; For the corresponding quantiles Trained independent lightweight temporal convolutional prediction network; The output is the fused feature sequence; Should Unique parameters of quantile networks.

5. The environmental dose dynamic prediction method based on multimodal edge fusion according to claim 1, characterized in that, Adaptive early warning is provided for the predicted dose rate in the radiation monitoring area at short future moments based on the dose rate prediction value, confidence interval, and warning threshold, including: The triggering conditions for the early warning mechanism are determined based on the dose rate prediction and confidence interval. These triggering conditions include attention triggering conditions, early warning triggering conditions, and alarm triggering conditions. The attention triggering conditions are as follows: The trigger condition indicates the predicted value of the median. Exceeding the warning threshold However, the lower limit of the confidence interval Not exceeding ; The warning trigger conditions are: The warning trigger condition represents the lower limit of the confidence interval. Exceeding the warning threshold ; Alarm trigger conditions: The alarm trigger condition indicates that the lower limit of the confidence interval exceeds the alarm threshold. , This represents two different thresholds; When the predicted dose rate of the radiation monitoring area at a future time meets the triggering conditions of the early warning mechanism, the corresponding early warning action will be executed.

6. A dynamic environmental dose prediction system based on multimodal edge fusion that implements the method of any one of claims 1-5, characterized in that, include: The first module is used to acquire multimodal data of the radiation monitoring area and synchronize it in time. The multimodal data includes radiation monitoring data, equipment status data and personnel video data. The second module is used to encode multimodal data according to the data structure to obtain encoded data; The third module is used to perform multimodal fusion on the encoded data using a multi-head attention mechanism to obtain fused data; The fourth module is used to perform prediction on the fused data using a lightweight temporal convolutional network to obtain the dose rate prediction value at future time. The fifth module is used to determine the confidence interval based on the dose rate prediction using the quantile regression method; The sixth module is used to provide adaptive early warning of the dose rate prediction value for the radiation monitoring area in the short term, based on the dose rate prediction value, confidence interval, and early warning threshold.

Images