Nuclear power unit power regulation scheme generation method and system based on deep reinforcement learning

Abstract

A kind of nuclear power unit power regulation control scheme automatic generation method and system based on deep reinforcement learning, by constructing the hybrid surrogate model of fusion CNN, GRU and self-attention mechanism, the dynamic response of reactor is represented, PPO-Lagrangian training strategy network is used, the accurate constraint control of axial power deviation in the regulation process of conventional AO method is realized;With current AO and target AO as input, the preset power and control rod position full-process operation sequence that meets the safety constraint is automatically generated;On the basis of existing operation, the remaining sequence is continued to be written, and the constraint violation caused by existing operation is repaired, the strategy network based on MLP introduces normalized space sampling to maintain training exploratory, hard constraint is used in test stage to ensure operation safety;Lagrangian multiplier self-adaptingly balances control performance and constraint satisfaction.The AO deviation can be controlled within the target value ±1.0%, and the generation time is less than 1 second after training is completed, which can provide real-time decision support for operators.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of nuclear reactor core control and operation automation, and particularly relates to a pressurized water reactor nuclear power unit fast power regulation control strategy automatic generation method and system based on deep reinforcement learning (DRL) and data-driven surrogate model, which is particularly suitable for realizing accurate, fast and safe constraint control of axial power offset (AO) under dynamic operating conditions such as load following, automatically generating preset power and control rod position operation sequences that meet nuclear safety requirements, and providing real-time decision support and emergency condition correction capability for operating personnel. BACKGROUND

[0002] Nuclear energy, as a clean, low-carbon and stable base load power source, plays an irreplaceable key role in the global energy structure transformation and "double carbon" target realization process. The safe operation and economic performance of nuclear reactors are directly related to the reliability of power grid power supply, energy costs and public acceptance of nuclear energy. Under the current background of new-type power system construction, the load fluctuation of power grid has significantly increased, which puts unprecedented requirements on the peak shaving capacity and operational flexibility of nuclear power units. In particular, under the important scenario of load following, the unit needs to adjust the power quickly and accurately according to the minute-level changes in power demand of the power grid, and the traditional base load operation mode has been difficult to adapt to the demand of modern power grid.

[0003] In the load following process, when the target power is given by the power grid dispatching instruction, the key control parameters of the reactor core need to be adjusted simultaneously to match the power demand. At present, this process highly depends on manual operation by the operating personnel: the operator needs to adjust multiple strongly coupled parameters such as control rod position and preset power according to experience, and continuously observe the changing trend of response parameters such as core axial offset (AO) and power distribution. Due to the high dimensionality of control parameters, the hysteresis of neutron dynamics response and the xenon poisoning feedback effect, the operator needs to wait for tens of seconds to minutes after a single adjustment to confirm the core response, resulting in a long time-consuming and heavy workload for the overall adjustment process, and there is a risk of human error. The traditional off-line simulation method relying on industrial software (such as core physics calculation program) takes several hours for a single power regulation calculation, which is difficult to complete multi-scheme comparison and boundary envelope analysis within the operating window, and cannot realize comprehensive coverage and real-time optimization for complex operating conditions.

[0004] The existing power regulation control technology has the following shortcomings: (1) The optimization method based on the physical model is time-consuming to calculate, and the calculation of a single power regulation operation sequence takes several hours, which cannot meet the online generation requirements; (2) Traditional PID or linear controllers are difficult to handle a wide range of nonlinear operating conditions and have poor robustness; (3) Rule-based expert systems rely on prior knowledge and cannot adaptively learn the optimal strategy; (4) In particular, when encountering safety constraint violations caused by human or program calculation errors during the power regulation process, it is impossible to quickly generate a reliable emergency correction strategy. Especially in the load tracking scenario, the existing technology is difficult to achieve second-level control scheme generation and adaptive optimization under the premise of ensuring safety constraints, which seriously restricts the ability of nuclear power units to participate in the rapid peak shaving of the power grid.

[0005] In recent years, reinforcement learning (RL) has shown potential in the control of complex systems, but its direct application to nuclear reactors faces two major challenges: first, the computational cost of core physics simulation is high, making trial-and-error training infeasible; second, control strategies must meet strict safety constraints, including axial offset limits, control saturation boundaries, and smoothness requirements. Therefore, there is an urgent need for a power regulation scheme generator that integrates high-fidelity substitution models and constrained reinforcement learning to achieve fast, safe, and adaptive closed-loop decision-making. Summary of the Invention

[0006] To address the shortcomings of existing nuclear power unit power reduction control methods, such as low computational efficiency, poor adaptability, and weak emergency response capabilities, this invention proposes a nuclear power unit power regulation scheme generator based on deep reinforcement learning. By constructing a data-driven core dynamics substitution model and employing constrained reinforcement learning (PPO-Lagrangian) for offline training of the agent, the generator can achieve the following functions during conventional AO (Automatic Aspect-Oriented) power regulation: automatically generating a full-process operation sequence that satisfies safety constraints for preset power and control rod positions; continuing to generate the remaining operation sequences based on existing operations; and simultaneously correcting constraint violations caused by existing operations. This invention can generate a scheme within 1 second, with the deviation between the AO and its target value controlled within ±1%, combining speed, safety, and economy.

[0007] This invention is achieved through the following technical solution:

[0008] A method for generating power regulation schemes for nuclear power units based on deep learning, characterized by the following steps:

[0009] S1. Establish a high-fidelity alternative model for the dynamic response of the nuclear reactor core power regulation process. This alternative model employs a hybrid neural network structure combining causal convolutional neural networks (CNN), recurrent neural networks (GRU), and self-attention mechanisms. The training set is time-series data generated by the core physics calculation program, and the current axial offset is used as the training data. , current control rod position , current preset power is input, and the predicted value of the axial offset at the next time is output;

[0010] S2. Construct a constrained reinforcement learning environment based on the surrogate model, define a state vector , wherein, is the target axial offset , define an action vector , wherein, is the fine-tuning amount of the preset power, is the adjustment amount of the control rod position;

[0011] Design a multi-objective reward function with a penalty , wherein the tracking reward , the boundary reward encourages the control amount to move away from the saturation region, and the smoothing reward ;

[0012] Constraint violation penalty , wherein , , , , is an adjustable coefficient, is a Lagrange multiplier, is a violation flag, is the preset power at the previous time, is the control rod position at the previous time; when the absolute deviation of the predicted axial offset at the next time and the target value is greater than a preset safety threshold, it is 1, otherwise it is 0;

[0013] S3. Adopt a proximal policy optimization-Lagrangian (PPO-Lagrangian) algorithm, train the policy network and the value network based on the environment established in step S2; specifically including:

[0014] a) Collect interaction trajectories based on the surrogate model;

[0015] b) Calculate the advantage function using the generalized advantage estimation algorithm;

[0016] c) Adopt a double-time-scale update mechanism, update the parameters of the policy network and the value network at a faster first time scale, and update the Lagrange multiplier λ based on the deviation gradient of the constraint violation rate at a slower second time scale, the Lagrange multiplier λ is used to dynamically adjust the strength of the constraint violation penalty term .​

[0017] S4. For a given initial axial offset and target axial offset, call the trained policy network to generate an action sequence; perform engineering processing on the action sequence, including power clipping, control rod position discretization (optional) and final state power matching, to form a directly executable power regulation operation sequence;

[0018] S5. Perform axial offset response verification on the power regulation operation sequence generated in step S4 using the surrogate model and / or the core physics calculation program; when the input part has been executed and results in an operation sequence that causes the axial offset to exceed the safety constraints, call the trained policy network to generate a subsequent correction operation sequence based on the current state, achieving rapid repair of constraint violations.

[0019] Further, the step S1 establishes a high-fidelity surrogate model of the dynamic response of the nuclear reactor core power regulation process, specifically including:

[0020] S1.1 Use the core physics calculation program CORCA-SIM to generate an action→state dataset for the power regulation task corresponding to the operating parameter domain, containing M≥6000 action-state time series samples generated by M different action sequences, each sample recording the sequence, M is the number of samples;

[0021] S1.2 Construct an autoregressive surrogate model based on causal convolutional CNN, GRU and Self-Attention for training and verification of deep reinforcement learning. The model input is the current axial offset , the current control rod position , the current preset power , and the output is the next time step axial offset , the cycle is repeated to complete the action→state modeling of the entire operation sequence. The single-step forward inference process of the model includes: causal convolutional CNN layer extracts local features of past actions, GRU captures and records long-range temporal dependencies, Self-Attention layer obtains the implicit relationship between GRU outputs at different time steps, and shallow MLP outputs single-step results. The model inference time is on the order of milliseconds, which is more suitable for fast training and iteration of deep reinforcement learning compared to the minute-level core physics calculation program.

[0022] Further, the method for updating the Lagrange multiplier λ in the second time scale in step S3 is: optimizing the objective function by gradient descent method, where is the average violation rate (averaged over different time steps), is the preset target violation rate.

[0023] Further, the step S4 is to perform the discretization rounding of the control rod position, which means that the continuous control rod position value output by the policy network is rounded according to the minimum moving step of the actual control rod drive mechanism (optional).

[0024] Further, the policy network outputs the probability distribution parameters of the action, and in the training stage, random sampling based on the distribution is used to maintain the exploration ability, and in the test or scheme generation stage, the expected value of the distribution is directly used as the deterministic action.

[0025] Secondly, the application also provides a nuclear power unit power regulation scheme generation system based on deep reinforcement learning, which is characterized by comprising:

[0026] a substitute model module, used for constructing the dynamic response high-fidelity substitute model;

[0027] a reinforcement learning intelligent agent module, comprising a trained policy network and a value network based on the PPO-Lagrangian algorithm, and capable of supporting the following functions:

[0028] (1) scheme generation and execution: used for generating a complete action sequence and performing engineering post-processing to output a directly executable power regulation operation sequence;

[0029] (2) scheme completion and emergency treatment: used for completing the first N-step action as a complete action sequence, and capable of generating a subsequent emergency correction scheme when a partial illegal operation sequence is input.

[0030] Further, the system is integrated into a nuclear power unit operator support system, a digital control system or a full-range simulator, and is used for providing real-time control decision, offline scheme pre-rehearsal or operation personnel training.

[0031] Compared with the prior art, the application has the following beneficial effects:

[0032] 1. Intelligent decision-making for the power reduction process of a nuclear power unit is realized

[0033] The application first applies deep reinforcement learning technology to the generation of power regulation control schemes for a nuclear power unit, realizes the intelligent decision-making change from artificial experience driving to data driving, and enables the system to automatically learn an optimal control strategy, generate a preset power and control rod position operation sequence meeting safety constraints, and significantly reduce the decision-making burden and operation risk of an operator. Experiments show that the generated scheme can control the deviation of AO from the target value within the range of ±1.0% of the target value, and the constraint violation rate is 0, which meets the strict requirements of nuclear safety operation.

[0034] 2. Efficient real-time scheme generation supporting multiple application scenarios

[0035] After the reinforcement learning agent is trained, the system can generate a complete power regulation operation scheme in 1 second, meeting the real-time requirements of online decision support. Compared with traditional offline calculation or lookup table methods, the present application has stronger flexibility and adaptability. The present application not only supports complete scheme generation, but also supports intelligent completion based on the first N steps of the action. This flexibility enables the system to be applied to various scenarios, including but not limited to: ① offline scheme library construction; ② online real-time decision support; ③ rapid response to emergency conditions; and ④ operation staff training simulation, and any existing method cannot reliably achieve all the above functions at the same time.

[0036] 3. Autoregressive hybrid surrogate model captures multi-scale dynamic characteristics and is compatible with reinforcement learning

[0037] The hybrid autoregressive surrogate model based on causal convolutional CNN, GRU and self-attention adopted by the present application can quickly simulate the dynamic response process of a nuclear reactor, replacing time-consuming physical calculations. The design combines the local feature extraction capability of the convolutional neural network, the long-term dependence modeling capability of the recurrent neural network and the global dependence capturing capability of the self-attention mechanism. In addition, the unique autoregressive structure of the surrogate model can be directly connected with the single-step control strategy of reinforcement learning, with a reasoning time of milliseconds, which is more suitable for the rapid trial-and-error training and iteration of deep reinforcement learning than the minute-level core physical calculation program.

[0038] 4. Strong scalability

[0039] The technical framework of the present application has good scalability. The surrogate model can be retrained according to the characteristics of different nuclear power units; the strategy network architecture can be flexibly adjusted according to the number of control variables; and the reward function can be customized according to different control targets. The method can also be applied to other power regulation methods, such as boron regulation and rapid regulation. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 is a comparison diagram of the AO response curve calculation results of the surrogate model and CORCA-SIM for the operation sequence of the test set;

[0041] Figure 2 is a diagram of the complete power regulation scheme generated by the reinforcement learning agent and the AO response curve calculated by the surrogate model;

[0042] Figure 3 is a comparison diagram of the AO response curve calculation results of the surrogate model and CORCA-SIM for the scheme generated by the agent;

[0043] Figure 4is the AO response curve calculation result schematic diagram of the emergency correction scheme of the existing scheme of the reinforcement learning agent responding to the upward exceeding of the safety constraint on CORCA-SIM;

[0044] Figure 5 is the AO response curve calculation result schematic diagram of the emergency correction scheme of the existing scheme of the reinforcement learning agent responding to the downward exceeding of the safety constraint on CORCA-SIM;

[0045] Figure 6 is a flowchart of a nuclear power unit power regulation scheme generation method based on deep learning;

[0046] Figure 7 is a schematic diagram of a substitute model architecture;

[0047] Figure 8 is a schematic diagram of the interaction process of the reinforcement learning agent and the substitute model. DETAILED DESCRIPTION

[0048] The technical solutions of the present application are further described below in combination with embodiments and drawings, but the protection scope of the present application should not be limited thereby.

[0049] As shown in Figure 6 , a nuclear power unit power regulation scheme generation method based on deep learning includes the following steps:

[0050] (1) Establish a high-fidelity substitute model of the core neutron dynamics of the power regulation process. Use the verified core physics calculation program CORCA-SIM to generate the action-state data set of the power regulation task corresponding to the operating parameter domain, a total of M>6000 T=12-step power regulation time sequence samples with different characteristics, each sample record complete sequence. Construct an autoregressive hybrid substitute model based on causal convolutional CNN, GRU and Self-Attention, in the embodiment, the input dimension is 3: , the model structure is as follows: the first layer of causal convolution uses a 3×1 convolution kernel, the channel number is 32, and the local space-time features of the control action are extracted; the second layer is a 2-layer GRU with a hidden layer dimension of 256, which captures long-range temporal dependence; the third layer introduces Multi-Head Self-Attention with 4 heads to model the nonlinear coupling effect between different time steps; finally, a 2-layer shallow MLP (256-128-1) is connected to output the next time step axial displacement prediction value . The root mean square error (RMSE) of the model single-step prediction reaches 0.06(%) on the validation set, and the mean absolute error (MAE) reaches 0.04(%) which is much smaller than the safety constraint of ±1%, sufficient to support engineering use.

[0051] (2) Determine the power regulation control target parameter: define the target axial offset , control step size , step length seconds, control variable rod position , preset power . In the embodiment, based on a typical 12-step load tracking procedure of a pressurized water reactor, the target axial offset , control step size T = 12 steps, and the decision interval of each step is minutes. The control variable constraint is that the single-step guide value during the preset power reduction process , the actual control amount ; the control rod bank position R ∈ [125, 225] steps. The disturbance interval is limited to steps, ensuring that the single-step regulation amplitude meets the response capability of the physical system.

[0052] (3) Construct a reinforcement learning state space. The state vector , where is the measured axial offset value at time t (unit: %), and is the preset target value.

[0053] (4) Construct a reinforcement learning action space. The action vector is a continuous fine-tuning amount, and the policy network outputs a Gaussian distribution , where is a learnable logarithmic standard deviation parameter. In the training phase, RSampling is used to directly sample continuous actions to ensure that the gradient is derivable; in the test phase, engineering processing is implemented, and the average action is directly used to meet the execution requirements of the DCS system.

[0054] (5) Design a multi-objective reward function with a penalty. In the embodiment, the specific expression is: , where the tracking reward ; the boundary reward encourages the control amount to be away from the saturation region; the smoothing reward suppresses high-frequency oscillation; and the constraint violation penalty , where is an adaptive Lagrange multiplier, is a violation flag, and when , the value of is 1, and otherwise, the value of is 0, and the threshold is 1%. The final reward value is clipped to the interval [-10, 2] to ensure training stability.

[0055] (6) Establish a PPO-Lagrangian constraint optimization framework. The policy network It is a 3-layer MLP (256-128-64), for state The mean of the Gaussian distribution of output actions With log standard deviation Lagrange multipliers Adaptively and dynamically adjust the constraint penalty strength to maximize the objective function. ,in For the dominant function, The pruning coefficients are used; the value network is a 3-layer MLP (128-128-1), for the state vector input... Output scalar Used to estimate the expected cumulative reward in the current state ,in Let be the discount factor, and minimize the loss function. In the embodiments ;

[0056] (7) Collect trajectories and calculate generalized dominance estimates. In each training round, trajectories are collected from a uniform distribution. Initial sampling Execution strategy Generate K=6 parallel trajectories with a time interval of T=12 steps. Generalized advantage estimation (GAE) is used. Among them, the timing difference error Count of synchronous constraint violations This is used for subsequent Lagrange multiplier updates.

[0057] (8) Update the policy network and Lagrange multipliers λ at different time scales. In the embodiment, the training process adopts a double-loop structure: the inner loop (each epoch) synchronously updates the policy network. and value network The AdamW optimizer is used, and the policy learning rate is... Value learning rate The outer loop performs Lagrange multiplier gradient descent every 5 epochs. ,in The average violation rate (averaged across different time steps). The target violation rate is set to 0.05. The λ update uses the Adam optimizer with a learning rate of... The learning rate is constrained to [0, 10] to prevent excessive penalty. When half of the total number of epochs has been trained (in this example), the learning rate is uniformly reduced to half of its original value to refine convergence.

[0058] (9) Post-processing of actions during the testing phase. Engineering constraints are applied to the continuous actions output by the strategy: In this embodiment, a preset power... The final step is mandatory To meet the final state requirements of power regulation; (optional) Control rod position A 5-step discretization process is implemented while satisfying the lower limit of the rod position.

[0059] (10) Generate and verify the complete power regulation curve. For the initial axial offset... Execute deterministic strategies Generate a T-step complete control sequence Simultaneously, the AO response curve corresponding to the strategy is calculated using an alternative model. Validation metrics include: average reward. Maximum dynamic deviation Number of violations Finally, the core physics calculation program CORCA-SIM was called to perform high-fidelity verification of the generated strategy.

[0060] (11) Verification of emergency correction and adjustment capabilities. For a given k-step control sequence that causes the AO to exceed the safety threshold. Calling the reinforcement learning agent to generate subsequent... The sequence of operations is calculated, and the corresponding AO response curves for the policy are computed using an alternative model. Validation metrics include the number of constraint violations after agent takeover. Finally, the core physics calculation program CORCA-SIM was called to perform high-fidelity verification of the generated strategy.

[0061] Example:

[0062] Taking a typical conventional AO power reduction operation as an example, this example requires the unit to reduce from full power to 80% FP within 2 hours. The reactivity management software needs to calculate and provide the control rod group position every 10 minutes during the conventional AO power adjustment process, and ensure that the core AO size is within ( ). Within ±1%), there were a total of 12 points, of which The method of using the nuclear power unit power regulation scheme generator based on reinforcement learning provided by the present invention in this embodiment includes the construction and training process of a high-fidelity substitution model, the construction and training process of a reinforcement learning agent, and the testing and verification process of the agent, referring to the aforementioned steps (1) to (11).

[0063] 1. Establish and train a high-fidelity alternative model of neutron dynamics in the reactor core according to step (1). Figure 1 The trained alternative model's predicted AO sequences for the operational sequences in the test set are shown and compared with the accurate results calculated by CORCA-SIM. The results show that the mean absolute error (MAE) of the alternative model is less than 0.04%, which is far less than the safety constraint of ±1%, sufficient for engineering use. The training process takes approximately 20-30 minutes.

[0064] 2. Establish and train the reinforcement learning agent according to steps (2) to (9). The training process takes about 10 minutes.

[0065] 3. Generate and verify the complete power regulation curve according to step (10). For the initial axial offset... Execute deterministic strategies Generate a complete 12-step control sequence Meanwhile, the AO response curve corresponding to the strategy is calculated using an alternative model. Figure 2 (The image shows a 12-step preset power adjustment scheme generated by the agent.) Figure 2 (Below) shows the 12-step R-group rod position adjustment scheme generated by the agent. Figure 2 (Above) shows the AO response curve calculated using the alternative model for this scheme. In the figure, the upper and lower red dashed lines represent the range of ±0.78%, the average reward. Maximum dynamic deviation Number of violations This demonstrates the superiority and high fault tolerance of intelligent agents.

[0066] 4. Call the core physics calculation program CORCA-SIM to perform high-fidelity verification of the generated complete strategy. This process takes less than 1 second. Figure 3 The complete agent generation scheme is presented, and the AO response curve calculated by CORCA-SIM is shown, with a comparison to the results of an alternative model. The results show that the high-fidelity verification results are very close to those of the alternative model, demonstrating not only the accuracy of the alternative model but also ensuring the real usability of the agent generation scheme.

[0067] 5. Verify the emergency correction and adjustment capability according to step (11). Conduct two sets of experiments: upward exceedance of safety constraints and downward exceedance of safety constraints. For both sets of experiments, the first 4 control sequences are manually entered, and then the intelligent agent takes over and fills in the last 8 control sequences. Finally, the core physics calculation program CORCA-SIM is called to perform high-fidelity verification of the generated scheme. Figure 4 , 5 The results of CORCA-SIM computations are presented, showing the agent's emergency correction and adjustment of existing schemes that exceed safety constraints both upwards and downwards. The results show that the agent can correct the AO to within the safety constraints within two steps and then maintain it within the safety constraints, demonstrating that the agent possesses both sufficient and rapid emergency handling capabilities, which is unmatched by any existing method. Furthermore, the alternative model can also provide relatively accurate results for schemes whose features differ significantly from both the training and test sets, demonstrating the robustness of the alternative model.

Claims

1. A method for generating power regulation schemes for nuclear power units based on deep learning, characterized in that, The steps include the following: S1. Establish a high-fidelity alternative model for the dynamic response of the nuclear reactor core power regulation process. This alternative model employs a hybrid neural network structure combining causal convolutional neural networks (CNN), recurrent neural networks (GRU), and self-attention mechanisms. The training set is time-series data generated by the core physics calculation program, and the current axial offset is used as the training data. Current control stick position Current preset power As input, output the axial offset at the next moment. The predicted value; S2. Construct a constrained reinforcement learning environment based on the aforementioned alternative model: Define state vector ,in, Offset along the target axis; Define action vectors ,in, This is a fine-tuning amount for the preset power. This is the amount of adjustment for the position of the control rod; Design a multi-objective reward function with penalties. Among them, tracking rewards Boundary rewards Encourage the control quantity to stay away from the saturation zone and smooth out rewards. Constraints and penalties for violations ,in , , , , This is an adjustable coefficient. For Lagrange multipliers, For violating the sign, The preset power from the previous moment. The value is the position of the control rod at the previous moment; when the absolute deviation between the predicted axial offset and the target value at the next moment is greater than the preset safety threshold, the value is 1, otherwise it is 0. S3. Using the Proximal Policy Optimization-Lagrangian (PPO-Lagrangian) algorithm, based on the environment established in step S2, the policy network is optimized. and value network The training includes: a) Collect interaction trajectories based on the aforementioned alternative model; b) Calculate the advantage function using the generalized advantage estimation algorithm. ; c) A dual-timescale update mechanism is adopted: the parameters of the policy network and the value network are updated at a faster first timescale, and the Lagrange multiplier λ is updated based on the deviation gradient of the constraint violation rate at a slower second timescale. The Lagrange multiplier λ is used to dynamically adjust the constraint violation penalty term. The strength; S4. Given an initial axial offset and a target axial offset, the trained policy network is invoked to generate an action sequence; the action sequence is then engineered, including power limiting, control rod position discretization, and final-state power matching, to form a directly executable power adjustment operation sequence. S5. Verify the axial offset response of the power regulation operation sequence generated in step S4 using the alternative model and / or the core physics calculation program; when the input part has been executed and the operation sequence that causes the axial offset to exceed the safety constraint is executed, call the trained policy network to generate a subsequent correction operation sequence based on the current state to achieve rapid repair of the constraint violation.

2. The method for generating a nuclear power unit power regulation scheme based on deep learning according to claim 1, characterized in that, Step S1 establishes a high-fidelity substitution model for the dynamic response of the nuclear reactor core power regulation process, specifically including: S1.1 The core physics calculation program CORCA-SIM is used to generate the action-state dataset corresponding to the operating parameter domain of the power regulation task. This dataset contains action-state time-series samples generated from M ≥ 6000 action sequences with distinct features. Each sample records... The sequence is M, where M is the number of samples; S1.2 Construct an autoregressive surrogate model based on causal convolutional CNN, GRU, and Self-Attention for training and validation of deep reinforcement learning. The model input is the current axial offset. Current control stick position Current preset power The output is the axial offset at the next moment. ,cycle The model completes the action-state modeling of the entire operation sequence in one step. The single-step forward inference process of the model includes: the causal convolutional CNN layer extracts local features of past actions, the GRU captures and records long-range temporal dependencies, the Self-Attention layer obtains the implicit relationship between the GRU outputs at different time steps, and the shallow MLP outputs the single-step result. The model inference time is in the millisecond range, which is more suitable for the rapid training and iteration of deep reinforcement learning compared to the minute-level core physics calculation program.

3. The method for generating nuclear power unit power regulation schemes based on deep learning according to claim 1, characterized in that, The method for updating the Lagrange multiplier λ in step S3 at the second time scale is as follows: optimize the objective function using gradient descent. ,in, For the average violation rate, The target violation rate is set in advance.

4. The method for generating a nuclear power unit power regulation scheme based on deep learning according to claim 1, characterized in that, In step S4, discretizing and rounding the control rod position means rounding the continuous control rod position values ​​output by the strategy network according to the minimum movement step of the actual control rod drive mechanism.

5. The method for generating nuclear power unit power regulation schemes based on deep learning according to claim 1, characterized in that, The policy network The probability distribution parameters of the output actions are used. During the training phase, random sampling based on this distribution is used to maintain the exploration capability. During the testing or solution generation phase, the expected value of this distribution is directly used as the deterministic action.

6. A nuclear power unit power regulation scheme generation system based on deep reinforcement learning, implementing the method described in any one of claims 1 to 5, characterized in that, include: The alternative model module is used to construct the dynamic response high-fidelity alternative model. The reinforcement learning agent module, including a trained policy network and a value network based on the PPO-Lagrangian algorithm, supports the following functions: (1) Scheme generation and execution: used to generate a complete action sequence and perform engineering post-processing to output a power regulation operation sequence that can be directly executed; (2) Solution completion and emergency handling: It is used to complete the first N steps of the executed action into a complete action sequence, and can generate subsequent emergency correction solutions when the input part violates the operation sequence.

7. The nuclear power unit power regulation scheme generation system based on deep reinforcement learning according to claim 6, characterized in that, The system is integrated into the nuclear power unit operator support system, digital control system, or full-range simulator to provide real-time control decisions, offline scenario simulations, or operator training.

Images