Nuclear emergency path and opportunity decision-making method based on reinforcement learning and deep learning

Abstract

The invention relates to a nuclear emergency path and opportunity decision-making method based on reinforcement learning and deep learning, and the method comprises the steps: taking minimization of exposure dose and evacuation time as targets, employing a deep reinforcement learning A3C algorithm to carry out the decision-making of the evacuation of a to-be-evacuated agent, and obtaining a plurality of evacuation paths and corresponding action opportunity schemes as candidate schemes; establishing a training set based on a decision sample formed by the candidate scheme, the corresponding decision variable and the multi-source environment information, training a deep learning model which is formed by Transform and a random forest and is from an environment state tensor to the candidate scheme, and outputting a prior score; and obtaining weighted scores of the candidate schemes by adopting linear weighting of score weights, obtaining a comprehensive score by fusing the weighted scores and the prior scores, and determining an executable scheme from the candidate schemes based on the comprehensive score. According to the method, the defects in the aspects of self-adaptability, real-time performance and benefit cost analysis capability in nuclear accident emergency decision-making and path planning in the prior art are overcome.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of nuclear emergency management and intelligent decision-making, and particularly relates to a nuclear emergency path and action timing decision-making method based on reinforcement learning and deep learning. BACKGROUND

[0002] Emergency response to nuclear power plant accidents needs to complete personnel evacuation or rescue in a timely manner while minimizing personnel radiation dose. Traditional technologies mainly focus on path planning optimization, generally assuming that environmental risks (such as radiation dose or dose rate) remain static or using average values, lacking consideration of dynamic changes. Especially in nuclear accident scenarios, radioactive plumes change over time and weather conditions, and the best action strategy may need to be adjusted according to real-time risks. For example, when the radiation field intensity suddenly increases at a certain moment, appropriate delay in departure or temporary avoidance during the journey may reduce the cumulative dose more than blind progress. However, existing evaluation systems have not established a quantitative analysis mechanism for the interaction between evacuation path planning and dynamic changes in the radiation field, lacking research on radiation dose evaluation and emergency strategies in dynamic emergency evacuation actions.

[0003] Some existing emergency decision support systems (such as nuclear accident consequence evaluation and decision support systems) consider multiple candidate paths and simple time factors, but generally only evaluate the dose and time consumption of different schemes through simulation, perform dose calculation and comparison based on preset schemes, and belong to the static analysis category, without involving deep reinforcement learning or real-time adjustment mechanism according to environmental feedback during the action process. There are many technical defects, specifically embodied in: the analysis of radiation dose mainly relies on the current dose value, which cannot adapt to the dynamically changing radiation environment, which may lead to emergency personnel being exposed to high dose areas, increasing the health risk. The cost analysis is limited to path length, simple time cost, and expected exposure dose, failing to comprehensively and fully consider the impact of road congestion and dynamically changing radiation environment on emergency personnel, leading to high risk and safety hazards in emergency decision-making. Under a large number of fixed rules, the decision-making process is complex, making it difficult to quickly generate action plans, affecting the efficiency of emergency response. SUMMARY

[0004] In view of the deficiencies of the prior art, the present application provides a nuclear emergency path and action timing decision-making method based on reinforcement learning and deep learning, which solves the deficiencies of the prior art in adaptability, real-time performance, and benefit-cost analysis capability in nuclear accident emergency decision-making and path planning, to make intelligent and efficient evacuation decisions under the influence of radiation factors.

[0005] The technical solution adopted by the present application is as follows: The present application provides a nuclear emergency path and timing decision-making method based on reinforcement learning and deep learning, comprising: Based on multi-source environmental information of a nuclear accident area, a deep reinforcement learning A3C algorithm is used to make decisions on evacuation of an evacuation agent to minimize exposure dose and evacuation time, and multiple evacuation paths and corresponding action timing schemes are obtained as candidate schemes; decision variables include a state space and an action space, the state space includes an environmental state tensor which includes current time step t downward radiation field, traffic conditions, agent position and shelter capacity, the action space includes movement direction of the current time step t , waiting time indicating whether to move immediately; a decision reward function , wherein , , is the path travel time of the time step t , the radiation dose received by the agent, and the congestion cost is the corresponding reward function weight ; a training set is established based on the candidate schemes, corresponding decision variables and multi-source environmental information, and a deep learning model is trained, which includes a Transformer and a random forest; the Transformer takes the environmental state tensor and prior control quantity as input, and takes the candidate schemes and corresponding summary features as output; the summary features include evaluation indexes of the schemes: total time, total radiation dose and total congestion cost; the prior control quantity is obtained by a parameter estimation module of the random forest from learning of the decision generated by the deep reinforcement learning A3C algorithm; the prior control quantity includes a waiting time threshold, a score weight corresponding to the evaluation index importance and the reward function weight, and a Transformer hyperparameter; a prior score module of the random forest pre-judges the candidate schemes output by the Transformer based on the summary features, and outputs a prior score ; each of the evaluation indexes is normalized, and the score weight is linearly weighted to obtain a weighted score of each candidate scheme, the weighted score and the prior score are fused to obtain a comprehensive score, and an executable scheme is determined from the candidate schemes based on the comprehensive score.

[0006] As a preferred technical solution: The parameter estimation module of the random forest takes macro scenario features contained in the multi-source environmental information as input, and takes the prior control quantity as output; the macro scenario features include meteorological conditions, accident source items, risk area, road network information and population density

[0007] The construction of the state space includes: The nuclear accident area is discretized into a grid of HxW, each grid node is a decision unit representing a road node or a block, and the environmental state tensor is defined at each node The radiation dose of each node is calculated by a nuclear pollution diffusion model; The traffic condition of each node is calculated by a traffic simulation model using traffic flow density and traffic capacity.

[0008] The construction of the action space includes: The agent moves along the nodes in sequence, and at a certain node, the decision action at the current time step t is determined. Wherein represents the next adjacent node number reached along the selected moving direction, represents the waiting time for going to the next adjacent node.

[0009] The executable scheme is determined from the candidate schemes based on the comprehensive score, which includes: According to the order of the comprehensive score from high to low, the M with higher scores are selected as the candidate schemes; The 0-1 linear constraint is performed on the candidate schemes by the shelter capacity, road capacity and time window to produce the highest utility feasible scheme, and the executable path action time sequence list is generated; if the constraint is not satisfied, the suboptimal feasible scheme is selected; The 0-1 linear constraint expression is as follows: In the formula, is the comprehensive score of the M th scheme in the i th scheme, is the constraint coefficient of each constraint.

[0010] The method further includes dynamic feedback and policy optimization using the deep reinforcement learning A3C algorithm and the deep learning model, which includes: Real-time acquisition of the current multi-source environmental information is performed to determine whether there is a mutation in the state space, and if there is, the deep reinforcement learning A3C algorithm is started for online fine-tuning, the weights of the corresponding items in the reward function are adjusted to make the calculation of the latest state quickly converge, and the Transformer and random forest are simultaneously started for local retraining to promote the deep reinforcement learning to prefer the evacuation path that can better cope with the mutation.

[0011] The dose safety check is performed on the current decision action to be executed, if the next state will exceed the dose threshold, the decision action is projected to the safe set and then executed; if it does not exceed the threshold, the original decision action is directly executed; when there is a strict dose budget, the Lagrange online adjustment is used to maintain the safety of the decision in an uncertain environment.

[0012] The method further includes: using the prior score to perform initial screening of the candidate solutions, deleting solutions with prior scores lower than a preset threshold, and retaining the remaining solutions as the basis for determining the executable solution.

[0013] The radiation dose received by the intelligent agent Calculate using the following formula: in, For the current time step t The location coordinates of the agent x , y Radiation dose at the location; The cost of congestion Calculate using the following formula: in, It was a step in time before t Traffic flow at the location of the intelligent agent. It refers to road capacity.

[0014] The Transformer employs an autoregressive decoder and uses attention rollout to synthesize an importance heatmap from multiple layers of multi-head attention to explain why a vehicle waits or detours at a certain location. The Transformer hyperparameters include beam search temperature and beam width.

[0015] The technical solution of the present invention can achieve at least some of the following beneficial effects: This invention employs the deep reinforcement learning A3C algorithm in conjunction with Transformer + Random Forest to not only plan the optimal emergency path but also determine when to begin action and when to continue or wait during the action. In other words, it integrates the path and the timing of action into a holistic decision-making problem for joint optimization. Through a deep reinforcement learning agent, it makes judgments as it goes, thereby dynamically adjusting the pace of action to adapt to real-time changes in environmental risks, achieving the comprehensive goal of reducing radiation exposure dose and shortening emergency response time.

[0016] Traditional emergency response plans follow a predefined "passive protection" paradigm, emphasizing execution according to predetermined paths and procedures, lacking the means to adjust in real time according to changing circumstances. The strategy proposed in this invention belongs to a "proactive decision-making" paradigm, where the agent can foresee and weigh future risk changes and make optimal choices in advance, such as proactively waiting or changing the pace. This paradigm shift significantly enhances the effectiveness and flexibility of emergency response.

[0017] This invention applies the deep reinforcement learning algorithm A3C to nuclear emergency decision-making, enabling the agent to handle dynamic decision-making problems with high-dimensional, continuous states. Simultaneously, a random forest is introduced as a meta-decision layer to achieve adaptive control of the deep reinforcement learning strategy, improving the model's adaptability to different scenarios and the reliability of its decisions. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the method of an embodiment of the present invention.

[0019] Figure 2 This is a schematic diagram of the action plan generated by the A3C algorithm of deep reinforcement learning in an embodiment of the present invention.

[0020] Figure 3 This is a schematic diagram of the structure of a deep learning model according to an embodiment of the present invention. Detailed Implementation

[0021] The specific embodiments of the present invention are described below with reference to the accompanying drawings.

[0022] See Figure 1 This embodiment of a nuclear emergency path and action timing decision-making method based on reinforcement learning and deep learning includes: S1. Collect multi-source environmental information of the nuclear accident area to construct the state space and action space of decision variables for nuclear emergency decision-making; based on the deep reinforcement learning A3C algorithm—that is, the asynchronous advantage Actor-Critic algorithm—with the goal of minimizing exposure dose and evacuation time, perform spatiotemporal linkage evacuation decision-making for the evacuation agents (personnel or convoys, etc.) to obtain multiple evacuation paths and corresponding action timing schemes as candidate schemes.

[0023] Specifically, during a nuclear emergency evacuation, the environment changes rapidly and is complex. Agents need to make swift decisions based on real-time environmental data (such as contamination spread, traffic congestion, and shelter capacity). This embodiment employs the deep reinforcement learning A3C algorithm, which trains multiple agents in parallel and guides the model to learn the optimal evacuation strategy through a dominance function. This strategy selects the optimal path and timing based on the current environmental state (such as radiation field, traffic conditions, and personnel locations) to minimize total exposure dose and evacuation time. This effectively addresses complex and dynamic environments.

[0024] The A3C deep reinforcement learning algorithm accelerates training through multi-threaded parallelism, where each thread has its own copy of the environment, policy network, and value network. Each thread's role can be divided into two parts: Actor and Critic. The actor is responsible for exploring the optimal policy (classification distribution) based on the current state. s t Output each action probability distribution And update the policy parameters through policy gradient. θ Critics then evaluate the effectiveness of each action, estimate the value of the current state, and calculate the advantage function as a reward signal for the actor's learning, thereby guiding the actor's learning process.

[0025] As a preferred approach, the multi-source environmental information includes accident source term information, real-time radiation monitoring, meteorological conditions, road network structure, population distribution, and shelter locations.

[0026] As a preferred approach, the construction of the state space includes: discretizing the nuclear accident area into an H×W grid (preferably set as H=W=101), with each grid node serving as a decision-making unit for a road node or block, and defining multi-channel environmental information at each node—radiation dose rate, road traffic status, agent location, and shelter capacity, etc., forming an environmental state tensor as the current time step. t environmental conditions s t At the same time, the location distribution and attributes of each intelligent agent to be evacuated are recorded.

[0027] Preferably, each node radiation dose The traffic conditions were calculated using a traffic simulation model based on vehicle flow density and capacity, obtained from a nuclear contamination diffusion model.

[0028] Specifically, environmental conditions In the formula: In the above formula, Q For the source term of radioactive materials, D The diffusion coefficient is... These are the coordinates of the nuclear radiation source. t It is time; Specifically, a traffic simulation model is used to calculate the traffic conditions at this node. : Among them, C This refers to the traffic capacity of this road node. This represents the traffic flow at that node. Specifically, the location of the intelligent agent It can be obtained through statistical data, while the capacity of shelters is determined by the region's emergency plan or infrastructure development plan.

[0029] The action space includes all possible actions that the agent can choose. As a preferred approach, the construction of the action space includes: the agent moving sequentially along nodes, and at a given node at the current time step... tDecision-making actions ,in This indicates the number of the next adjacent node to be reached along the selected direction of movement. This indicates the waiting time before proceeding to the next adjacent node. The movement direction includes east, south, west, north, and 0 (no movement). The waiting time indicates whether to move immediately. If not moving immediately, the user waits at the current position, and this value is positive. If moving immediately, this value is 0.

[0030] Once the state space and action space are constructed, the deep reinforcement learning A3C algorithm can be started for evacuation training. The A3C algorithm's actor network... Based on the current state, each thread autonomously searches for a suitable evacuation path and timing, and each thread records its state-action sequence. Upload to the global network and share update policy network parameters. and value network parameters A3C uses the advantage function A. To improve learning efficiency. The training process is as follows: (1) Initialization: Initialize the environment, define the state space and action space, and randomly initialize the parameters of the policy network and value network. (2) Environment interaction: Each thread selects an action according to the policy network and interacts with the environment to obtain the state of the next time step. and instant rewards (3) Calculate the advantage: Using the current policy and the critic network, calculate the advantage function about the current state. (4) Policy gradient update: Update the parameters of the policy network according to the advantage function. (5) Critics update: Update the parameters of the critic network by minimizing the mean squared error. (6) Model update and feedback: Pass the updated parameters to other threads for asynchronous training. Throughout the training process, by continuously updating the network parameters, the agent gradually learns how to choose the optimal withdrawal path and action timing.

[0031] Specifically, the decision reward function in this embodiment ,in, , , For time step t The path travel time, the radiation dose received by the intelligent agent, and the cost of congestion. This represents the weight of the corresponding reward function.

[0032] In this embodiment, the congestion cost represents traffic conditions, and the calculation formula is described above.

[0033] in, .

[0034] Through extensive iterative training, the deep reinforcement learning A3C algorithm can automatically learn effective evacuation decision-making strategies. After training, multiple evacuation routes and corresponding departure timing schemes can be generated by sampling from the final policy network. These schemes simultaneously consider the changes in environmental pollution over time and space, achieving spatiotemporal coordinated planning of routes and timing.

[0035] In this specific simulation scenario, this step assumes that the nuclear contamination source is located in the city center of area A, and the contamination diffusion model is based on the existing Lagrange particle diffusion model. The following are the assumed data parameters: (1) Pollution source parameters: Release type: Point source continuous release.

[0036] Total release: Q = 1 × 10 10 Bq (total amount of radioactive material released during a nuclear accident).

[0037] Initial pollution source release location ( , = (50, 50) (the center point of the road network in City A, i.e., the grid center point).

[0038] (2) Traffic flow and capacity: The capacity of the road network is randomly set. Between vehicles per hour.

[0039] Traffic flow is handled using a simple traffic flow model. ,in This is the flow coefficient.

[0040] (3) Population density and shelter capacity: Population distribution is set based on population density statistics for City A, with the initial population of each node randomly set between 100 and 10,000. Shelter locations and capacities are set using emergency planning data, with capacities ranging from 200 to 500 people. The number of selected agents can be set based on training cases (1-100).

[0041] This step generates multiple action plans, each with its own specific effects, as follows: Figure 2 As shown, the scheme sets up 9 action entities represented by different colors. Each action entity represents an action team (the number of people can be set according to the population distribution of each point). The dots represent the positions that each action entity has moved to at the current time. The solid lines in the figure represent the paths that the action entities have already traversed in the case. The yellow pentagrams at the bottom of the map represent the action endpoint of the scheme. The heat map in the figure shows the pollutant diffusion at different times (time steps) simulated in the scheme.

[0042] S2. Learning planning strategies based on Transformer + Random Forest (see [link]). Figure 3 Specifically, it includes: A training set is established based on the candidate schemes and the corresponding decision variables and decision samples composed of multi-source environmental information, and a deep learning model is trained, which includes Transformer and Random Forest. The Transformer takes the environmental state tensor and prior control variables as inputs and the candidate schemes and corresponding summary features as outputs; the summary features include the scheme evaluation indicators: total time T, total radiation dose D, and total congestion cost C; The prior control quantity is learned by the parameter estimation module of the random forest from the decisions generated by the deep reinforcement learning A3C algorithm. This prior control quantity is mainly used to control the decoding of the Transformer and the scoring weights of subsequent steps. Preferably, it includes a waiting time threshold, a weight representing the importance of each evaluation indicator, a scoring weight corresponding to the reward function weight, and Transformer hyperparameters; the Transformer hyperparameters are the bundle search temperature (T) and bundle width (B). Specifically, the parameter estimation module (RF-Param) of the random forest takes the macroscopic scene features contained in the multi-source environmental information as input and the prior control quantity as output; the macroscopic scene features include meteorological conditions, accident source terms, risk area, road network information, and population density. The prior scoring module (RF-Score) of the random forest predicts the candidate solutions output by the Transformer based on the summary features and outputs a prior score.

[0043] In the aforementioned deep learning model, the deep reinforcement learning A3C algorithm provides training samples and policy distribution (for Transformer distillation), optimal hyperparameter labels (for RF-Param learning), and candidate quality labels (for RF-Score learning). During the online phase, if replanning is triggered, A3C can be fine-tuned in short steps, or a fast link relying solely on Transformer+RF can be used to complete updates within seconds.

[0044] The deep learning model specifically uses the environment state encoding of the candidate schemes obtained in S1 as the input sequence to train a sequence-to-sequence model similar to GPT / Transformer. Its input is an environment state tensor or its feature representation, and its output is a path-timing decision sequence. The Transformer preferably employs an autoregressive decoder with three parallel output heads: an action classification head, a timing regression head, and a risk or dose assistance head. To accommodate the subsequent multi-objective ranking in S3, a beam search + temperature sampling + diversity penalty is used. For each time step, a loss combining imitation learning + distillation + structure regularization is applied. in, For Huber's losses; Used to encourage the generation of diverse candidates; For A3C or simulation-recorded unit step dose labels; The loss weights are (preferred values ​​are shown in Table 1).

[0045] In the specific simulation scenario, the training process and hyperparameters for this step are as follows: Reasoning process and complexity / time estimation process: (1) Take the most recent data from the monitoring / forecasting system The state tensor at each time step; (2) RF-Param estimation of inference priors (weights, temperature, threshold); (3) The Transformer autoregressive generates k “paths + timings”; (4) RF-Score performs prior fast scoring and outputs the candidate set to S3.

[0046] Hyperparameters: Complexity: Single-layer self-attention Total complexity In this embodiment, we take... With a CPU environment beamwidth B=8 and candidate schemes K=6, the test showed that the candidate set can be output within 1-2 seconds, and the GPU environment can also achieve similar results. Second.

[0047] Optimizer: AdamW; Learning rate: 2 × 10⁻⁶ -4 (Including cosine annealing); Batch size: 32; Training rounds: 30; Early stop: Stop if the verification loss does not decrease after 5 rounds.

[0048] Regular expression: Dropout 0.1; weight decay 1 × 10 -2 .

[0049] Time window: (Last 6 time steps); Total token length: The beam width is B=8 and the number of candidates is K=6.

[0050] RF: Number of trees M=200, maximum depth 12, minimum number of leaf samples 5, OOB error monitoring.

[0051] Attention interpretability: To demonstrate "why wait / detour at a certain node," attention rollout is used to synthesize an importance heatmap by combining multiple layers of multi-head attention. The layer attention matrix is (After residuals and normalization), then: in, It is the first The effective attention matrix after residual mixing of layer attention. This is the cross-layer cumulative attention matrix. It is the total number of attention layers participating in Rollout. Given the length of the input token sequence, Let w be the one-hot indicative vector at time t, and w be the importance distribution for all input tokens. Mapping the tokens back to the grid yields a spatial-temporal importance map. If high weights are concentrated on "the dose in this road segment will reach its peak in 3 minutes," then the "wait" decision at that node has an interpretable basis. This represents the residual ratio; a value of 0.2–0.3 is recommended. Key parameter values ​​are shown in Table 1 below.

[0052] Table 1. Key Parameter Diagram

[0053] Preferably, the training process specifically includes: S21. Compile the convergent decision samples of the deep reinforcement learning A3C algorithm in S1 under multiple scenarios (weather, source terms, traffic, population) into a supervised dataset: (1) State sequence Within the sliding time window (most recent) The environmental tensor sequence at each time step , denoted as: in, The length parameter of the sliding time window. C represents the set of state sequences of a training sample within a time window, where C×H×W represents the dimension of the node's environment state tensor.

[0054] (2) Tag sequence (Derived from A3C trajectory): Summary features including movement actions and waiting times for each step, as well as phased arrival times, cumulative dose (total time T, total radiation dose D, total congestion cost C), etc. The planned time-domain length of the label sequence is represented by the upper limit of the number of steps for the actions to be generated and the waiting time, which is a positive integer (determining the truncation length of the supervision label and the distillation sequence). For time step The discrete actions are taken from a finite set of actions. For time step The duration or waiting time is a non-negative real number. Zero indicates immediate movement, while a positive value indicates waiting for the specified duration at the current position before moving.

[0055] (3) Strategy distribution (for distillation): A3C's strategy output This formula is the policy distribution learned by A3C, in the state Next action The probability distribution is used for policy distillation and as a soft label to guide Transformer decoding (it can also be used as a priori reference when online).

[0056] Data splitting (example): Training / Validation / Test = 70% / 15% / 15%; Sample deduplication and time window overlap are controlled to <50% to avoid information leakage.

[0057] S22. Input Representation and Action Labeling (1) Spatiotemporal embedding: Patch / tokenize: Will After spatial slicing into S, it is then spliced ​​together over time to obtain... .in This represents the number of spatial patches into which a single frame is divided. This represents the total length of the token sequence.

[0058] Two-dimensional spatial location encoding is superimposed on each token. With time and location encoding : in, This is a token representation that combines spatial location encoding and temporal location encoding.

[0059] (2) Action and timing discretization / regression composite annotation: Classification tags are formed based on action sets. Waiting time It can be discretized into several bins (0, 1, 2, 5 minutes...) or performed as a continuous regression (a hybrid head is recommended: classification + regression).

[0060] S3. Strategy thrust and multi-objective ranking decision-making, which includes: The result of S2 K The evaluation indicators of each candidate solution are normalized, and the weighted scores of each candidate solution are obtained by linear weighting using the scoring weights. The weighted scores and the prior scores are combined to obtain a comprehensive score. Based on the comprehensive score, an executable solution is determined from the candidate solutions.

[0061] Specifically, among the evaluation indicators, the first... i Total evacuation time for all options: ; Total cumulative dose ; Total cost of congestion: .

[0062] In the above formula, L i Representing the The number of time steps for each candidate solution (i.e., the length of the action sequence of the solution). Represents a continuous-time variable, used for integration over a single time step interval, with a range of values. , Represents the time step index. It is the first The plan is in the first The duration or waiting time of each time step. It is the first The plan is in the first Representative spatial locations at each time step (such as the grid center corresponding to a path segment or a road segment sampling point) are used to read values ​​from the environmental channel.

[0063] As a preferred method, S3 includes the following steps: S31. Use range normalization: in, For the first The first scheme The standardized value obtained by normalizing the range of the index values ​​is in the range [0,1], representing the relative size of the index. For the first The first scheme Class indicator value, It is a very small constant (e.g., 10). -5 Or 10 -6 This is used to avoid division by zero errors and ensure stable calculations.

[0064] S32. Use the scoring weights given by RF-Param for total evacuation time, total radiation dose, and total congestion cost. ( Linear weighting is performed to obtain the weighted score: In the above formula, , , For the original indicators The normalized value.

[0065] S33. Will The prior score output by the deep learning model The results are obtained by merging the scores. : In the above formula, These are weighting coefficients, which can be fixed (e.g., 0.35) or adaptively adjusted by RF-Param. This is the prior score output by the deep learning model. This score is obtained by the model based on historical training data and represents an initial prediction score for the solution under the current environmental conditions. When the scenario risk increases, The adjustment is made to strengthen a priori robustness.

[0066] S34. Based on the comprehensive score, determine the executable solutions from the candidate solutions, including: Sort according to overall score, and select the one with the higher score. M (Preferably 1 to 3) options are selected; The candidate solutions are subjected to 0–1 linear constraints based on shelter capacity, road segment traffic capacity, and time window to perform feasibility screening and conflict resolution, producing the most efficient feasible solution and generating an executable path action sequence list; if the constraints are not met, the second-best feasible solution is selected. The expression for the 0–1 linear constraint is as follows: In the formula, for M In the first scheme i The overall score of the proposed solutions The constraint coefficient is the constraint coefficient for each constraint.

[0067] As a preferred embodiment, S3 also includes: S35. The candidate solutions are initially screened using the prior score, and solutions with prior scores lower than a preset threshold are deleted, while the remaining solutions are retained as the basis for determining the executable solution.

[0068] As a preferred embodiment, the method further includes: S4. Utilizing the aforementioned deep reinforcement learning A3C algorithm and deep learning model for dynamic feedback and policy optimization, which includes: The system collects current multi-source environmental information in real time, determines whether there is a sudden change in the state space, and if so, starts the deep reinforcement learning A3C algorithm for online fine-tuning, adjusts the weights of the corresponding terms in the reward function to make the calculation of the latest state converge quickly, and simultaneously starts Transformer and Random Forest for local retraining, so that the deep reinforcement learning is biased towards an evacuation path that can better cope with sudden changes.

[0069] As a preferred method, S4 specifically includes the following steps: S41. Use EWMA to detect mutations and trigger reprogramming: in, It is a time step The state value, These are weighting coefficients, with values ​​ranging from [0,1]. For the current time step The original state values ​​are usually certain quantities in the environment (such as time, radiation dose, traffic conditions, etc.). For state The expected value (usually a steady-state or initial value) is used to measure the difference between the current state and the desired state. It is a state The standard deviation reflects the fluctuation range of the current state. The threshold coefficient for triggering replanning is controlled if the state change exceeds the expected value. of Double standard deviation This will trigger a replanning process.

[0070] S42. Adaptive Weighting and Rewards: The S3 scoring weights are dynamically adjusted synchronously as the situation changes. The reward function weights of the deep reinforcement learning A3C algorithm in S2 () ): In the above formula, the superscript ' represents the adjusted parameter. These are adjusted weighting coefficients, used to adjust the weight of each score based on the current situation characteristics. These are the original weighting coefficients, representing the weight of each objective in the initial environment. It is a coefficient that is dynamically adjusted based on the current environmental conditions (such as radiation intensity, traffic flow, etc.). It is a function used to adjust the weights according to specific environmental conditions (such as high radiation areas or traffic congestion). The adjusted scoring weights represent the current situation. T , D , C The weighting coefficients for the three objectives, The original scoring weights represent the basic importance of each objective (evacuation time, radiation dose, congestion cost). It is an adjustment function for each objective based on the current environmental situation, indicating how the weight of the objective is adjusted according to different environmental changes. It is the weighted sum of all objectives, used to ensure the normalization of the weights so that the adjusted sum of weights is still 1.

[0071] S43. Safety Fence (Action Projection): The decision action to be executed is dose-safe. If the next state is predicted to exceed the dose threshold, the decision action is projected onto the safety set before execution. If the threshold is not exceeded, the original decision action is executed directly. When there is a strict dose budget, online Lagrange adjustment is used to maintain the safety of the decision in uncertain environments.

[0072] The security set is defined as follows: In the above formula, This is the radiation dose threshold; "Pass" = 1 means immediate action, i.e., no waiting. For a moment The safe set is defined as the next safe state if the predicted radiation level at the current location is below a threshold and the road segment is passable at this moment. This is a projected safe action. When the originally planned action falls into the unsafe set in the next state, the action is "projected" to the safe set that is closest to the original action before execution. This indicates that the original plan was at time The actions to be performed are those given by the policy before security projection is performed. It is the environment transition function. Given the current state. and candidate actions Predict the next state position and related environmental quantities after performing the action, and use this information to determine whether the action belongs to a safe set. , N , E , S , WThese represent the north, east, south, and west directions, respectively. 0 means not to move and wait for an opportunity to move.

[0073] In the above dynamic feedback and strategy optimization process, a fast re-inference and rolling execution scheme is preferably adopted—rolling in a "short cycle—small step size" manner (e.g., every 1-2 minutes): (1) Rerun S2 lightweight inference with the latest settings (beam width / number of candidates can be reduced); (2) Use S3 for rapid scoring and constraint verification; (3) Only perform the first step of the current cycle and proceed to the next cycle.

[0074] In the above dynamic feedback and strategy optimization process, dose budget constraints are preferably applied, when a strict dose budget exists. At that time, Lagrange online regulation was adopted: In the above formula: Strategy (a mapping from state to action); : Lagrange multiplier (penalty factor for dose expectation); : Raw real-time report (negative values ​​include weighted factors such as time, dosage, and congestion); : No. The actual dose increment per step (or current cycle) (consistent with the dose increment definition in S41). : The expected total dose; : Total number of steps or total duration of the assessment (used to distribute the total budget across each step or cycle); The observed current average dose (which can be the average of the last few steps or a sliding window, or an instantaneous estimate). ); Step size (the update speed of the two variables); In strategy Expectation under induced trajectory distribution; The operator that projects onto the non-negative half-axis is defined as follows: .

[0075] Step S3 of this embodiment, multi-objective ranking and executable constraints, achieves unified scoring of candidate solutions based on multiple objectives such as time, dosage, and congestion. It also combines constraints such as capacity, traffic flow, and connectivity to output a directly executable "path-action schedule." Building on this, step S4, with its runtime based on online monitoring-triggered weight adaptation and short-step fine-tuning, combined with action safety projection and rolling replanning, ensures safety, stability, and high timeliness even under sudden situational changes. Step S4 allows for continuous maintenance of safety, controllability, and optimal timeliness in uncertain real-world environments. Combined with the robust ranking and executable constraints of S3, this forms a closed-loop nuclear emergency joint decision-making system.

[0076] It will be understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A nuclear emergency path and timing decision-making method based on reinforcement learning and deep learning, characterized in that, include: Based on multi-source environmental information of the nuclear accident area, with the goal of minimizing exposure dose and evacuation time, the deep reinforcement learning A3C algorithm is used to make evacuation decisions for the intelligent agents to be evacuated, and multiple evacuation paths and corresponding action timing schemes are obtained as candidate schemes. Decision variables include a state space and an action space. The state space includes an environment state tensor, which includes the current time step. t The action space includes the current time step, the radiation field, traffic conditions, agent location, and shelter capacity. t The direction of movement, the waiting time representing whether to move immediately; the decision reward function. ,in, , , For time step t The path travel time, the radiation dose received by the intelligent agent, and the cost of congestion. These are the corresponding reward function weights; A training set is established based on the candidate schemes and corresponding decision variables, along with decision samples composed of multi-source environmental information. A deep learning model, including Transformer and Random Forest, is trained on this model. The Transformer takes the environmental state tensor and prior control variables as input, and the candidate schemes and corresponding summary features as output. The summary features include evaluation metrics for the schemes: total time, total radiation dose, and total congestion cost. The prior control variables are learned by the parameter estimation module of the Random Forest from decisions generated by the deep reinforcement learning A3C algorithm. The prior control variables include a waiting time threshold, a weight representing the importance of each evaluation metric, a scoring weight corresponding to the reward function weight, and Transformer hyperparameters. The prior scoring module of the Random Forest predicts the candidate schemes output by the Transformer based on the summary features and outputs a prior score. The evaluation indicators are normalized, and the weighted scores of each candidate solution are obtained by linear weighting using the scoring weights. The weighted scores and the prior scores are then combined to obtain a comprehensive score. Based on the comprehensive score, an executable solution is determined from the candidate solutions.

2. The method according to claim 1, characterized in that, The parameter estimation module of the random forest takes the macroscopic scene features contained in the multi-source environmental information as input and the prior control quantity as output; the macroscopic scene features include meteorological conditions, accident source terms, risk area, road network information, and population density.

3. The method according to claim 1, characterized in that, The construction of the state space includes: The nuclear accident area is discretized into an H×W grid, with each grid node representing a decision unit for a road node or block, and the environmental state tensor is defined on each node. The radiation dose at each node was calculated using a nuclear contamination diffusion model; The traffic conditions at each node are calculated by the traffic simulation model using traffic density and capacity.

4. The method according to claim 3, characterized in that, The construction of the action space includes: The agent moves sequentially along the nodes, and at a certain node at the current time step... t Decision-making actions ,in This indicates the number of the next adjacent node to be reached along the selected direction of movement. This indicates the waiting time before proceeding to the next adjacent node.

5. The method according to claim 1, characterized in that, The step of determining an executable solution from candidate solutions based on a comprehensive score includes: Sort according to overall score, and select the one with the higher score. M One is considered as a candidate option; The candidate solutions are subjected to 0–1 linear constraints based on shelter capacity, road segment traffic capacity, and time window to perform feasibility screening and conflict resolution, producing the most efficient feasible solution and generating an executable path action sequence list; if the constraints are not met, the second-best feasible solution is selected. The expression for the 0–1 linear constraint is as follows: In the formula, for M In the first scheme i The overall score of the proposed solutions The constraint coefficient is the constraint coefficient for each constraint.

6. The method according to claim 1, characterized in that, The method further includes using the deep reinforcement learning A3C algorithm and deep learning model for dynamic feedback and policy optimization, which includes: The system collects current multi-source environmental information in real time, determines whether there is a sudden change in the state space, and if so, starts the deep reinforcement learning A3C algorithm for online fine-tuning, adjusts the weights of the corresponding terms in the reward function to make the calculation of the latest state converge quickly, and simultaneously starts Transformer and Random Forest for local retraining, so that the deep reinforcement learning is biased towards an evacuation path that can better cope with sudden changes.

7. The method according to claim 6, characterized in that, The decision action to be executed is dose-safe. If the next state is predicted to exceed the dose threshold, the decision action is projected onto the safety set before execution. If the threshold is not exceeded, the original decision action is executed directly. When there is a strict dose budget, online Lagrange adjustment is used to maintain the safety of the decision in uncertain environments.

8. The method according to claim 1, characterized in that, The method further includes: using the prior score to perform initial screening of the candidate solutions, deleting solutions with prior scores lower than a preset threshold, and retaining the remaining solutions as the basis for determining the executable solution.

9. The method according to claim 1, characterized in that, The radiation dose received by the intelligent agent Calculate using the following formula: in, For the current time step t The location coordinates of the agent x , y Radiation dose at the location; The cost of congestion Calculate using the following formula: in, It was a step in time before t Traffic flow at the location of the intelligent agent. It refers to road capacity.

10. The method according to claim 1, characterized in that, The Transformer employs an autoregressive decoder and uses attention rollout to synthesize an importance heatmap from multiple layers of multi-head attention to explain why a vehicle waits or detours at a certain location. The Transformer hyperparameters include beam search temperature and beam width.

Images