A tokamak long-pulse plasma shape control method

Abstract

The application discloses a tokamak long-pulse plasma shape control method, comprising the following steps: acquiring real-time state information in the operation process of a tokamak device, wherein the real-time state information at least comprises plasma shape parameters, current safety margins of each poloidal field coil and historical time sequence state data; inputting the real-time state information into a pre-trained reinforcement learning intelligent agent; outputting voltage control amounts corresponding to each poloidal field coil by the reinforcement learning intelligent agent; and applying the voltage control amounts to corresponding poloidal field coils to realize closed-loop plasma shape control. The application realizes active margin management of multiple poloidal field coils in the voltage mode, effectively suppresses the overcurrent risk of single coils or multiple coils, and improves the steady-state operation capability and system robustness of long-pulse discharge in the voltage control mode of the tokamak.

Description

Technical Field

[0001] This invention relates to the field of plasma configuration control technology, and in particular to a tokamak long-pulse plasma configuration control method, apparatus, device, and storage medium. Background Technology

[0002] A tokamak is a device that uses a toroidal magnetic field to confine a high-temperature plasma to achieve controlled nuclear fusion. Its stable operation is highly dependent on the precise control of the plasma. Traditional control methods often face bottlenecks such as high computational complexity, limited adaptive capability, and high system integration cost when dealing with strong coupling of multiple variables, nonlinear dynamics, and sudden disturbances.

[0003] Currently, intelligent control methods based on deep reinforcement learning are gradually being applied to the field of tokamak plasma magnetic control, effectively improving the autonomous control capability of plasma configuration. Tokamak poloidal coil control is mainly divided into current control mode and voltage control mode. The voltage control mode eliminates the inner-loop current control stage, allowing direct regulation of the coil voltage and current change rate, resulting in faster response speed, stronger dynamic control capability, and better suitability for rapid real-time plasma configuration control scenarios.

[0004] However, the voltage control mode has inherent safety defects: in voltage mode, the coil current is the integral accumulation of voltage and cannot be directly constrained by control commands. There is no explicit current closed-loop limit mechanism. During long pulse discharges of tens to hundreds of seconds, even if the plasma configuration control accuracy remains good, any overcurrent in any unipolar field coil will trigger hardware protection and forcibly terminate the entire discharge process, which severely restricts the long pulse steady-state discharge capability.

[0005] Most existing reinforcement learning control schemes are designed for short-pulse discharge conditions and are based on current control mode for strategy training and verification. Current mode can directly constrain coil current through commands, which can implicitly avoid overcurrent risks. Therefore, existing technologies do not address the long-term dynamic scheduling and active overcurrent prevention control of multi-coil current margin in voltage mode. They lack a constraint mechanism for the safety margin of multi-coil current and cannot achieve adaptive load distribution and overcurrent risk prediction and avoidance in long-pulse voltage mode conditions. It is difficult to balance high configuration control accuracy and long-pulse stable discharge requirements. Summary of the Invention

[0006] This invention provides a tokamak long-pulse plasma configuration control method, which realizes active margin management of multi-pole field coils in voltage mode, effectively suppresses the overcurrent risk of single or multiple coils, and takes into account both high configuration control accuracy and the requirement of stable long-pulse discharge.

[0007] In a first aspect, embodiments of the present invention provide a tokamak long-pulse plasma configuration control method, comprising: The real-time status information during the operation of the tokamak device is obtained, and the real-time status information includes at least plasma configuration parameters, current safety margin of each poloidal field coil, and historical time-series status data. The real-time state information is input into a pre-trained reinforcement learning agent, which then outputs the voltage control quantity corresponding to each pole field coil. The voltage control quantity is applied to the corresponding poloidal field coils to achieve plasma closed-loop configuration control.

[0008] Furthermore, the normalized value range of the current safety margin is [-1, 1]. The sign of the margin represents the direction in which the coil current approaches the risk of overcurrent, and the absolute value of the margin represents the urgency of the overcurrent risk. Among them, a margin of 0 indicates that the coil current is in the center of the safe range, a margin of 1 indicates that a positive overcurrent has occurred, and a margin of -1 indicates that a negative overcurrent has occurred.

[0009] Furthermore, the training phase of the reinforcement learning agent employs a multi-objective composite reward function, which includes a tracking reward term, an action penalty term, and a coil current overcurrent prevention penalty term. The tracking reward term is used to generate a reward based on the tracking error between the actual values ​​of plasma current and configuration parameters and the reference target values, so as to drive the plasma state to converge toward the target configuration. The action penalty term is used to impose penalty constraints on the amplitude and variation range of the voltage control action output by the intelligent agent, so as to obtain a smooth and controllable control output. The coil current overcurrent prevention penalty term is constructed based on the current safety margin of all poloidal field coils. It applies a progressively increasing penalty to coils that approach the overcurrent threshold. When multiple coils approach the threshold simultaneously, the penalties are superimposed, guiding the agent to achieve active allocation and management of the current margin of multiple coils.

[0010] Furthermore, the reinforcement learning agent adopts an Actor-Critic network architecture and completes policy iterative training based on a proximal policy optimization algorithm.

[0011] Furthermore, the voltage control quantities corresponding to each poloidal field coil output by the reinforcement learning agent include: Based on the real-time status information input, the safety margin of the coil current of each poloidal field coil and the plasma configuration control target are comprehensively weighed. When the current of any poloidal field coil is detected to be approaching the safety limit, the agent autonomously lowers the voltage control quantity corresponding to that coil, while raising the voltage control quantity corresponding to the other poloidal field coils with residual current margin, thereby avoiding the risk of coil overcurrent while maintaining the configuration control target.

[0012] Furthermore, the training process of the agent adopts a course learning strategy, gradually increasing the difficulty of controlling the rate of change of plasma current during the training process, and gradually tightening the current safety margin constraint boundary of each pole field coil, so as to progressively improve the agent's multi-coil collaborative compensation control capability under overcurrent boundary conditions and external strong disturbance conditions.

[0013] Furthermore, the method also includes: The voltage control quantity output by the intelligent agent is gain-scaled using a safe voltage limit gain vector so that the scaled voltage control quantity is limited to the safe voltage range allowed by the corresponding coil power supply hardware.

[0014] In a second aspect, embodiments of the present invention provide a tokamak long-pulse plasma configuration control device, comprising: The status acquisition module is used to acquire real-time status information during the operation of the tokamak device. The real-time status information includes at least plasma configuration parameters, current safety margin of each poloidal field coil, and historical time-series status data. The reinforcement learning module is used to input the real-time state information into a pre-trained reinforcement learning agent, and the reinforcement learning agent outputs the voltage control quantity corresponding to each pole field coil. The configuration control module is used to apply the voltage control quantity to the corresponding poloidal field coils to achieve plasma closed-loop configuration control.

[0015] Thirdly, embodiments of the present invention provide an electronic device, comprising: Memory, used to store computer programs; A processor for executing the computer program; Wherein, when the processor executes the computer program, it implements the tokamak long-pulse plasma configuration control method described in any of the first aspects above.

[0016] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed, implements the tokamak long-pulse plasma configuration control method described in any of the first aspects above.

[0017] Compared with existing technologies, the tokamak long-pulse plasma configuration control method provided by this invention has the following advantages: It acquires real-time state information during the operation of the tokamak device, including at least plasma configuration parameters, current safety margins of each poloidal field coil, and historical time-series state data; it inputs the real-time state information into a pre-trained reinforcement learning agent, which outputs voltage control quantities corresponding to each poloidal field coil; and it applies the voltage control quantities to the corresponding poloidal field coils to achieve closed-loop plasma configuration control. This invention achieves active margin management of multi-poloidal field coils in voltage mode, effectively suppressing the overcurrent risk of single or multiple coils, and improving the steady-state operation capability and system robustness of long-pulse discharge in tokamak voltage control mode. Attached Figure Description

[0018] To more clearly illustrate the technical features of the embodiments of the present invention, the drawings used in the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic flowchart of a tokamak long-pulse plasma configuration control method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the training loop architecture of the reinforcement learning agent provided in an embodiment of the present invention; Figure 3 This is a graph showing the change rate of plasma current with the number of training epochs during the training process, provided in an embodiment of the present invention. Figure 4 This is a first control effect diagram of the tokamak long-pulse plasma configuration control method provided in the embodiment of the present invention; Figure 5 This is a second control effect diagram of the tokamak long-pulse plasma configuration control method provided in the embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of a tokamak long-pulse plasma configuration control device provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to limit the invention.

[0023] In a first aspect, embodiments of the present invention provide a tokamak long-pulse plasma configuration control method, see [link to previous document]. Figure 1 This is a flowchart illustrating an embodiment of a tokamak long-pulse plasma configuration control method provided by the present invention.

[0024] like Figure 1 As shown, the method includes the following steps: S1: Obtain real-time status information during the operation of the tokamak device. The real-time status information includes at least plasma configuration parameters, current safety margin of each poloidal field coil, and historical time-series status data. Specifically, within each control cycle, the plasma state observation vector output by the real-time diagnostic system of the acquisition device includes at least the normalized plasma current error and the position errors of multiple preset plasma boundary control points. The position errors of the boundary control points specifically include the magnetic flux difference of the controlled point and the magnetic field strength at point X, and also include the current safety margin PFRM corresponding to all controlled poloidal field PF coils.

[0025] After obtaining the observation vector at a single moment, the observation vectors at the current control moment and those from N consecutive control cycles backward are spliced ​​and flattened in chronological order to form the final state representation of the reinforcement learning agent. The stacked historical step size N is determined based on the characteristic time constant τ of magnetic field penetration in the tokamak vacuum chamber, satisfying the constraint N·Δt≥τ, where Δt is the closed-loop control cycle of the device.

[0026] By constructing states through historical stacking, the agent is provided with time-series observation information spanning multiple control cycles. This allows the agent to implicitly perceive the dynamic evolution of eddy currents inside the vacuum chamber and capture the changing trends of currents in each pole field coil. This provides a complete time-series information foundation for the agent to conduct inference decisions on multi-coil collaborative overcurrent prevention and dynamic distribution of coil loads.

[0027] S2: Input the real-time state information into the pre-trained reinforcement learning agent, and the reinforcement learning agent outputs the voltage control quantity corresponding to each pole field coil; Specifically, reinforcement learning agents are pre-trained, and multiple independent simulation environment replicas are created using parallel computing tools. These multiple simulation environment replicas can run synchronously and in parallel, each independently completing environmental interaction, state sampling, and sample collection, and collecting training sample data in parallel.

[0028] After the training iteration converges and the agent training is completed, it is connected to the tokamak real-time control system. The trained reinforcement learning agent is deployed into the real-time plasma control system (PCS) of the tokamak device. The real-time state information obtained in step S1 is input into the deployed reinforcement learning agent, and the agent generates the final voltage control quantity.

[0029] S3: Apply the voltage control quantity to the corresponding poloidal field coils to achieve plasma closed-loop configuration control.

[0030] Specifically, the voltage control quantity is sent to the power supply unit corresponding to the poloidal field coil. After receiving the voltage command, each power supply outputs a driving voltage of the corresponding amplitude and applies it to both ends of the poloidal field coil.

[0031] In summary, this invention selects voltage control mode to achieve a higher response speed than current command mode. It addresses the issue of coil current overcurrent in voltage mode through a current safety margin mechanism. This allows the intelligent agent to proactively reduce control requirements on a single coil when it faces overcurrent risk, while automatically calling other coils with sufficient margin for precise compensation. This achieves proactive margin management of multi-pole field coils in voltage mode. Without sacrificing plasma configuration control accuracy throughout the process, it effectively suppresses the risk of overcurrent in single or multiple coils, significantly reduces forced shutdowns caused by coil current exceeding limits during long-pulse discharge, and significantly improves the steady-state operation capability and system robustness of long-pulse discharge in tokamak voltage control mode.

[0032] In one optional implementation, the normalized value range of the current safety margin is [-1, 1]. The sign of the margin represents the direction in which the coil current approaches the risk of overcurrent, and the absolute value of the margin represents the urgency of the overcurrent risk. Among them, a margin of 0 indicates that the coil current is in the center of the safe range, a margin of 1 indicates that a positive overcurrent has occurred, and a margin of -1 indicates that a negative overcurrent has occurred.

[0033] Specifically, The formula for calculating the current safety margin is as follows: ; in, This is the overcurrent limit value for the poloidal field coil. This refers to the real-time coil current value obtained from the coil acquisition. This represents the current margin that can be further increased in the positive direction of the coil.

[0034] The current margin is then input into a custom Softplus piecewise function to map and transform the physical quantity margin value to the normalized interval [0,1]. The expression of this custom Softplus function is as follows: ; in, This is the lower limit threshold of the margin. This is the upper limit threshold for the remaining margin; when the remaining margin is less than... This indicates that the positive adjustable margin is very limited, and the function outputs a fixed value of 1. When the remaining margin is greater than 1, the function will output a fixed value of 1. This indicates that the forward current adjustment margin is very sufficient. The function output is a fixed value of 0. The exponential expression operation is performed within the upper and lower threshold range to ensure that the output result falls within the numerical range of [0,1].

[0035] Since the coil is susceptible to overcurrent in both positive and negative directions, it is necessary to distinguish between the positive and negative directions of the current and to determine the appropriate direction based on the real-time coil current. The symbol is used to perform absolute value correlation operations on the Softplus output to obtain the final PFRM item.

[0036] The sign of the current safety margin PFRM obtained after normalization indicates the direction of the overcurrent risk, and its absolute value indicates the urgency of the overcurrent risk. PFRM=0 indicates that the coil current is in the center of the safe range and the margin is the most sufficient. PFRM→+1 indicates that the coil current is increasing positively and approaching the positive overcurrent limit. PFRM=+1 indicates that a positive overcurrent has occurred. PFRM→-1 indicates that the coil current is increasing negatively and approaching the negative overcurrent limit. PFRM=-1 indicates that a negative overcurrent has occurred.

[0037] During the long-pulse discharge operation of the tokamak, the normalized current safety margin corresponding to each poloidal field coil is calculated in real time. The current safety margin of all coils is fed into the reinforcement learning agent as a component of the state observation. At the same time, the normalized current safety margin is introduced into the reward function. Based on the normalized value, the reinforcement learning agent perceives the risk direction and the urgency of risk of each coil. When the absolute value of the current safety margin of some coils continues to increase, that is, when the coil gradually approaches the current limit in the corresponding direction, the agent actively reduces the voltage output demand of such coils. At the same time, it schedules the output compensation voltage of other poloidal field coils with smaller absolute values ​​of current safety margin and sufficient adjustment margin, thereby dispersing the coil control load while ensuring that the plasma configuration tracking accuracy does not deteriorate significantly.

[0038] In one optional implementation, the training phase of the reinforcement learning agent employs a multi-objective composite reward function, which includes a tracking reward, an action penalty, and a coil current overcurrent prevention penalty. The tracking reward term is used to generate a reward based on the tracking error between the actual values ​​of plasma current and configuration parameters and the reference target values, so as to drive the plasma state to converge toward the target configuration. The action penalty term is used to impose penalty constraints on the amplitude and variation range of the voltage control action output by the intelligent agent, so as to obtain a smooth and controllable control output. The coil current overcurrent prevention penalty term is constructed based on the current safety margin of all poloidal field coils. It applies a progressively increasing penalty to coils that approach the overcurrent threshold. When multiple coils approach the threshold simultaneously, the penalties are superimposed, guiding the agent to achieve active allocation and management of the current margin of multiple coils.

[0039] Specifically, the multi-objective composite reward function is obtained by a weighted combination of three parts: a tracking reward term, an action penalty term, and a coil current overcurrent prevention penalty term. The overall expression of the reward function is: ; in, This is the total reward value for a single step. To track reward items, To track the weight coefficient groups corresponding to the reward items, For action-based penalties, This is the weight coefficient group corresponding to the action penalty item. This is a penalty item for preventing overcurrent in the coil. This is the total weighting coefficient for the coil current overcurrent prevention penalty term.

[0040] Tracking reward items This is used to evaluate the deviation of the actual plasma operating state from the target reference state. The input includes three types of error signals: magnetic flux error at eight plasma boundary control points. Radial and axial magnetic field errors at point X and plasma current tracking error The expression is: ; The weighting coefficient group contains three sub-weights: α, β, and γ, which correspond to the weights of magnetic flux error, X-point magnetic field error, and plasma current error, respectively, and satisfy the normalization constraint α+β+γ=1.

[0041] Action Punishment The voltage control action used to constrain the agent's output includes two parts of penalty, expressed as follows: ; in, For the control action of the current control cycle, For the control action of the previous control cycle, the first penalty term is the sum of squared differences between adjacent cycle actions, used to penalize instantaneous jumps in the control action, and the second penalty term is the sum of squared action amplitudes, used to penalize excessive overall amplitude of the control action.

[0042] The weight coefficient group includes δ and ε, which correspond to the weights of the jump penalty and the magnitude penalty, respectively.

[0043] Overcurrent prevention penalty for coil current Using the normalized current safety margin PFRM of all poloidal field coils as input, the expression is: ; Multi-coil risk information fusion is achieved using the Smoothmax operator. The specific calculation formula for the Smoothmax operator is as follows: ; in, For the input sequence, in this embodiment, it is the current safety margin (PFRM) of each poloidal field coil. The weights for each input item can be differentiated based on the hardware characteristics of different poloidal field coils. The sharpness coefficient is the sharpness factor. The larger the value, the closer the calculation result is to the direct maximum value of the input. i is the coil number, and n is the total number of poloidal field coils involved in the calculation.

[0044] When the absolute value of any PFRM approaches 1, indicating that the coil has entered the overcurrent warning range, a progressively increasing penalty is applied. When the absolute values ​​of the PFRMs of multiple coils approach 1 simultaneously, the penalties are cumulative. This design guides the agent to learn to maintain overall control accuracy by coordinating precise compensation of other coils while reducing reliance on the near-limit coil, thereby preventing overcurrent.

[0045] The multi-objective composite reward function constructed in this embodiment integrates plasma configuration tracking, control output smoothness, and multi-coil overcurrent risk constraints into the optimization objectives of reinforcement learning. It can automatically identify the coil with the highest risk and form a progressive penalty. The trained agent no longer simply pursues configuration tracking indicators, but learns to actively carry out multi-coil load transfer and compensation while maintaining configuration control accuracy. It realizes active management of coil current margin in voltage mode from the training mechanism level.

[0046] In one alternative implementation, the reinforcement learning agent employs an Actor-Critic network architecture and completes iterative policy training based on a proximal policy optimization algorithm.

[0047] Specifically, the intelligent agent adopts an asymmetric neural network structure, consisting of two parts: an actor network and a critic network. The critic network is more complex and has a stronger ability to learn complex value functions, while the actor network remains lightweight when deployed on the EAST plasma control system (PCS), meeting the requirements for inference speed.

[0048] The Critic network is used to evaluate the value of a state to guide policy updates. It is used only during the training phase. It consists of three hidden layers, each with 256 neurons. All neurons use the tanh activation function. The output layer consists of a single neuron that outputs the state value v of the current state.

[0049] The Actor network performs policy reasoning. After offline training, it is deployed in the EAST plasma control system (PCS) to perform real-time control reasoning. It receives historical stacked state observation vectors as input and passes them through two fully connected hidden layers with 128 and 64 neurons respectively. Both hidden layers use the tanh activation function, and the output layer also uses the tanh activation function, with a 12-dimensional output dimension. This corresponds to the action output of the 12 poloidal PF coils of the EAST device, resulting in a 12-dimensional raw action quantity. This raw action quantity is finally converted into the final voltage control quantity that can be sent to the hardware.

[0050] For example, see Figure 2 This is a schematic diagram of the training loop architecture for reinforcement learning agents.

[0051] The dashed box on the right side of the figure contains the interaction module between the intelligent agent and the simulation environment. The Actor module includes the Controller, which is the actor network under training. The Environment is a copy of the parallel simulation environment, which can deploy multiple instances for parallel sampling. The Targets module provides various reference target values ​​t such as plasma current and configuration. The Mapping module completes the calculation and construction of state and reward.

[0052] The interaction process is as follows: The Controller outputs action 'a' and sends it to the Environment simulation environment. The simulation environment outputs raw measurement information 'm' and passes it to the Mapping module. The Mapping module receives the target reference quantity 't' from the Targets, combines it with the measurement information 'm' output by the environment, completes error calculation, coil PFRM calculation, and historical stacking of observation vectors, and generates state 's' and single-step reward 'r'. State 's' is sent back to the Controller as the next network input. The reward 'r', the current state 's', and action 'a' together form a sample tuple {a,s,r} and output it to the Experience Replay unit on the left.

[0053] The experience replay unit stores a large number of {a,s,r} training samples obtained through interactive acquisition. Then, it extracts the samples according to the sampling logic of the PPO algorithm and sends them back to the Controller on the Actor side through the Update link to complete the iterative update of the parameters of the actor network and the critic network, thus forming a complete closed-loop training cycle. With the help of the multi-replica Environment, a large number of interactive samples can be generated in parallel, improving the efficiency of training data acquisition and accelerating policy convergence.

[0054] In one optional implementation, the voltage control quantities corresponding to each poloidal field coil output by the reinforcement learning agent include: Based on the real-time status information input, the safety margin of the coil current of each poloidal field coil and the plasma configuration control target are comprehensively weighed. When the current of any poloidal field coil is detected to be approaching the safety limit, the agent autonomously lowers the voltage control quantity corresponding to that coil, while raising the voltage control quantity corresponding to the other poloidal field coils with residual current margin, thereby avoiding the risk of coil overcurrent while maintaining the configuration control target.

[0055] Specifically, after receiving real-time status information including plasma configuration parameters, normalized current safety margins of each coil, and historical timing information, the intelligent agent no longer generates control commands solely based on plasma configuration tracking errors. Instead, it simultaneously weighs the global plasma configuration control accuracy target and the real-time current safety margin constraints of each poloidal field coil to achieve coupled decision-making between control targets and hardware safety constraints.

[0056] During real-time inference, the agent determines the current margin and overcurrent risk level of each poloidal field coil in real time based on the magnitude and direction of the current safety margin of each coil. When it detects that the current of any poloidal field coil is approaching the positive or negative safety limit and entering the high-risk operating range, the agent will actively reduce the original voltage output of the high-risk coil to reduce the dependence on the control of the coil with the adjacent limit and avoid the continuous accumulation of coil current exceeding the limit. At the same time, the agent adaptively increases the voltage output of the other poloidal field coils with sufficient current margin, and uses the control capability of the surplus coils to dynamically compensate for the control attenuation of the high-risk coil. Through the coordinated load redistribution among multiple coils, the magnetic field control deviation caused by the reduction of the load of a single coil is offset, ensuring that the overall plasma boundary morphology, magnetic flux distribution and plasma current always conform to the target control state.

[0057] This embodiment realizes active margin management of multi-pole field coils in voltage mode. Without sacrificing the plasma configuration control accuracy throughout the process, it effectively suppresses the risk of overcurrent in single or multiple coils, significantly reduces the forced shutdown phenomenon caused by coil current exceeding the limit during long pulse discharge, and significantly improves the steady-state operation capability and system robustness of long pulse discharge in tokamak voltage control mode.

[0058] In one alternative implementation, the training process of the agent adopts a course learning strategy, gradually increasing the difficulty of controlling the rate of change of plasma current during the training process, and gradually tightening the current safety margin constraint boundary of each pole field coil, thereby progressively improving the agent's multi-coil collaborative compensation control capability under overcurrent boundary conditions and external strong disturbance conditions.

[0059] Specifically, the training process gradually increases the difficulty of the task by progressively increasing the difficulty of controlling the rate of change of plasma current. Simultaneously, the current safety margin constraints of each poloidal field coil are dynamically adjusted to progressively improve the agent's multi-coil collaborative compensation control capability under overcurrent boundary conditions and strong external disturbance conditions. The training process variation curve corresponding to this embodiment is shown in the figure below. Figure 3 As shown.

[0060] Figure 3This is a graph showing the change rate of plasma current with the number of training epochs during the course learning and training process in this embodiment. The horizontal axis represents the number of training epochs, and the vertical axis represents the change rate of plasma current, in kA / s. In the early stage of training, when the number of training epochs is less than 400, the applied change rate of plasma current is maintained at a low level. When the number of training epochs exceeds 400, the average value and random value range of the current change rate gradually increase in sync, with the peak value of the current change rate reaching a maximum of 240 kA / s. When the number of training epochs reaches 1000, the average value and value range of the current change rate tend to stabilize, and the agent continues to carry out strategy iteration within this high change rate range.

[0061] While applying a random rate of change to the target current, a margin random perturbation mechanism is also set for the poloidal field coils during the training phase: four poloidal field coils are randomly selected in each training round and assigned random current safety margin margins. In the simple learning phase with fewer than 400 training rounds, the constraint boundary of the coil current safety margin is set relatively strictly. When the number of training rounds exceeds 400 and the rate of change of plasma current begins to gradually increase, the constraint margin of the coil current safety margin is adaptively relaxed simultaneously.

[0062] The entire training process follows a learning logic from easy to difficult. The agent first learns basic configuration control capabilities under simple samples with low current change rate and strict constraint boundaries, and then gradually encounters complex operating condition samples with high current change rate and random disturbances in coil margin distance.

[0063] This embodiment adopts a phased training scheme of course learning, which enables the agent to gradually master the configuration control capability under the condition of large current change rate. At the same time, it fully learns the load transfer and collaborative compensation logic when the coil is close to the overcurrent boundary, thereby improving the agent's generalization ability in the face of random coil margin disturbances and strong current dynamic disturbances.

[0064] In an optional implementation, the method further includes: The voltage control quantity output by the intelligent agent is gain-scaled using a safe voltage limit gain vector so that the scaled voltage control quantity is limited to the safe voltage range allowed by the corresponding coil power supply hardware.

[0065] Specifically, the safety voltage limit gain vector contains gain components that correspond one-to-one with each poloidal field coil. The value of each gain component is set to 80% of the engineering safety voltage limit of the corresponding poloidal field coil. The original voltage action output by the agent is multiplied component by component with the safety voltage limit gain vector to complete the scaling transformation, and the final voltage command sent to the coil power supply is obtained.

[0066] This embodiment ensures, from the control structure level, that the voltage command issued to the poloidal coil power supply is always within the safe range that the hardware can withstand by setting a safe voltage limit gain vector, thus avoiding the generation of over-range voltage commands due to abnormal output of the intelligent agent strategy.

[0067] In one optional implementation, this embodiment presents the actual control results of the EAST device #159696 discharge experiment, verifying the actual control effect of the long-pulse plasma configuration control method of the present invention. The experimental results are shown below. Figure 4 , Figure 5 .

[0068] Figure 4 This is a schematic diagram of the coil current evolution under EAST#159696 discharge RL control conditions. Figure 4 The top shows the currents of each poloidal field coil. The curves show the changes over time, with the vertical axis in kA and the horizontal axis representing time [s]. Different colored curves correspond to the six poloidal field coils PF1-PF6. Figure 4 Below are the current safety margin (PFRM) timing curves for each coil. The PFRM value range is [-1, 1]. The closer the PFRM is to -1, the closer the negative current of the coil is to the overcurrent limit. The two red dashed lines on the left and right of the figure correspond to the controller switching time t=1.1s and the discharge end time t≈20s, respectively.

[0069] Figure 5 This is a schematic diagram illustrating the evolution of key plasma parameters under EAST#159696 discharge RL control conditions. From top to bottom, they represent: ISO FLUX error (in Wb) for multiple controlled plasma boundary segments, used to characterize plasma boundary shape tracking deviation; magnetic fields BrX and BzX at point X (in T), reflecting the position control effect at point X; and plasma current, with the blue curve representing the target plasma current Ip. target The orange curve represents the actual plasma current Ip, in mA; the plasma elongation ratio is kappa; the horizontal axis is also time [s], and the red dashed lines mark the controller switching time and the discharge termination time, respectively.

[0070] In this discharge experiment, at t=1.1s, the control of the device was directly switched from the original PID controller to the RL controller of this invention. The controller switching process did not include a transition phase. Figure 5It can be seen that after switching the controller, key configuration parameters such as plasma boundary flux error, X-point magnetic field, plasma current, and elongation ratio can quickly converge and stably track the target reference value. No obvious plasma disturbance was induced at the moment of switching, and no plasma rupture occurred throughout the process. The total duration of this discharge was about 20 seconds, and it went through three typical operating conditions completely, verifying the control capability of multi-coil cooperative overcurrent prevention.

[0071] The first phase lasts from 1.1 seconds to 10 seconds, corresponding to... Figure 4 , Figure 5 Within the 1.1s-10s time interval, the plasma current, boundary magnetic flux, X-point position, and elongation ratio all stably follow the target curve, and the configuration tracking error remains at a low level. Figure 4 As can be seen from the PFRM curves below, the PFRM values ​​of each PF coil remain around 0, indicating that the bidirectional current regulation margin of each coil is sufficient and there is no risk of near-overcurrent in the coil.

[0072] The second stage lasts from 10s to 17s. As the discharge continues to evolve, the negative current in the PF4 coil increases, and the corresponding PFRM gradually approaches -1. After t=12.5s, the PFRM of the PF2 coil also approaches -1. The RL agent senses the overcurrent risk of the coils through the PFRM observation signal and executes the control logic of "reducing the voltage demand of high-risk coils + compensating for other coils": actively reducing the voltage output of PF4 and PF2 to reduce the dependence on the regulation of these two adjacent limit coils, while automatically increasing the voltage output of coils such as PF1 and PF6, whose PFRM is still close to 0 and have sufficient adjustment margin, to achieve a redistribution of the regulation load among the coils. Figure 5 It is evident that the tracking accuracy of various plasma configuration parameters did not show any observable deterioration during this stage, ensuring the quality of configuration control while suppressing the risk of coil overcurrent.

[0073] The third stage lasts 17-20 seconds, marking the later stage of discharge. Figure 4 As shown in the PFRM curve below, during the 17.5s-20s period, the PFRM of multiple PF coils simultaneously approached -1, and the control margin of multiple coils synchronously approached the hardware limit. The RL agent continued to solve for the voltage distribution balance between the plasma configuration tracking target and the overcurrent constraint of multiple coils, trying its best to maintain the discharge operation until the device's volt-second count was exhausted, the PFRM of multiple coils simultaneously reached the limit, the overall physical design limit of the system was broken, and the discharge ended. Throughout the entire approximately 20s continuous RL control process, the plasma did not break up, verifying the engineering feasibility of this scheme under voltage-mode long-pulse conditions.

[0074] This embodiment demonstrates, based on actual discharge experimental results from the EAST device, that the reinforcement learning controller of this invention can achieve seamless direct switching to replace the original PID controller, and the switching process will not disturb the plasma operating state. When some coils gradually approach the overcurrent limit, the agent can rely on the PFRM to perceive the risk and autonomously complete the coil load transfer and compensation control, taking into account both configuration tracking accuracy and coil overcurrent protection requirements. The discharge only ends when multiple coils simultaneously exhaust the control margin and reach the physical limit of the system, avoiding the defect of forced discharge interruption due to single coil overcurrent in traditional voltage mode. This effectively improves the steady-state operation capability of long-pulse discharge in tokamak under voltage mode, providing a feasible engineering solution for intelligent magnetic control of long-pulse plasma in nuclear fusion devices.

[0075] Secondly, embodiments of the present invention provide a tokamak long-pulse plasma configuration control device, see [link to related document]. Figure 6 This is a schematic diagram of an embodiment of a tokamak long-pulse plasma configuration control device provided by the present invention.

[0076] like Figure 6 As shown, the device includes: The status acquisition module 21 is used to acquire real-time status information during the operation of the tokamak device. The real-time status information includes at least plasma configuration parameters, current safety margin of each poloidal field coil, and historical time-series status data. The reinforcement learning module 22 is used to input the real-time state information into the pre-trained reinforcement learning agent, and the reinforcement learning agent outputs the voltage control quantity corresponding to each pole field coil. Configuration control module 23 is used to apply the voltage control quantity to the corresponding poloidal field coils to realize plasma closed-loop configuration control.

[0077] In one optional implementation, the normalized value range of the current safety margin is [-1, 1]. The sign of the margin represents the direction in which the coil current approaches the risk of overcurrent, and the absolute value of the margin represents the urgency of the overcurrent risk. Among them, a margin of 0 indicates that the coil current is in the center of the safe range, a margin of 1 indicates that a positive overcurrent has occurred, and a margin of -1 indicates that a negative overcurrent has occurred.

[0078] In one optional implementation, the training phase of the reinforcement learning agent employs a multi-objective composite reward function, which includes a tracking reward, an action penalty, and a coil current overcurrent prevention penalty. The tracking reward term is used to generate a reward based on the tracking error between the actual values ​​of plasma current and configuration parameters and the reference target values, so as to drive the plasma state to converge toward the target configuration. The action penalty term is used to impose penalty constraints on the amplitude and variation range of the voltage control action output by the intelligent agent, so as to obtain a smooth and controllable control output. The coil current overcurrent prevention penalty term is constructed based on the current safety margin of all poloidal field coils. It applies a progressively increasing penalty to coils that approach the overcurrent threshold. When multiple coils approach the threshold simultaneously, the penalties are superimposed, guiding the agent to achieve active allocation and management of the current margin of multiple coils.

[0079] In one alternative implementation, the reinforcement learning agent employs an Actor-Critic network architecture and completes iterative policy training based on a proximal policy optimization algorithm.

[0080] In one optional implementation, the voltage control quantities corresponding to each poloidal field coil output by the reinforcement learning agent include: Based on the real-time status information input, the safety margin of the coil current of each poloidal field coil and the plasma configuration control target are comprehensively weighed. When the current of any poloidal field coil is detected to be approaching the safety limit, the agent autonomously lowers the voltage control quantity corresponding to that coil, while raising the voltage control quantity corresponding to the other poloidal field coils with residual current margin, thereby avoiding the risk of coil overcurrent while maintaining the configuration control target.

[0081] In one alternative implementation, the training process of the agent adopts a course learning strategy. During the training process, the difficulty of controlling the rate of change of plasma current is gradually increased, and the current safety margin constraint boundary of each pole field coil is gradually tightened, so as to progressively improve the multi-coil collaborative compensation control capability of the agent under overcurrent boundary conditions and external strong disturbance conditions.

[0082] In an optional embodiment, the device is further configured to: The voltage control quantity output by the intelligent agent is gain-scaled using a safe voltage limit gain vector so that the scaled voltage control quantity is limited to the safe voltage range allowed by the corresponding coil power supply hardware.

[0083] It should be noted that the tokamak long-pulse plasma configuration control device provided in this embodiment of the invention is used to execute all the process steps of the tokamak long-pulse plasma configuration control method of the above embodiment. The working principle and beneficial effects of the two are one-to-one, so they will not be described again.

[0084] Thirdly, embodiments of the present invention provide an electronic device, see [link to previous document]. Figure 3 The diagram shown is a structural schematic of an electronic device provided in an embodiment of the present invention.

[0085] like Figure 7 As shown, the device includes: Memory 31 is used to store computer programs; Processor 32 is used to execute the computer program; When the processor 32 executes the computer program, it implements the tokamak long-pulse plasma configuration control method as described in any of the above embodiments.

[0086] For example, the computer program may be divided into one or more modules / units, which are stored in the memory 31 and executed by the processor 32 to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the electronic device.

[0087] The processor 32 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0088] The memory 31 can be used to store the computer programs and / or modules. The processor 32 implements various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory 31 and calling the data stored in the memory 31. The memory 31 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 31 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital card (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0089] It should be noted that the aforementioned electronic devices include, but are not limited to, processors and memory, as will be understood by those skilled in the art. Figure 7 The structural diagram is merely an example of the electronic device described above and does not constitute a limitation on the electronic device. It may include more components than shown in the diagram, or combine certain components, or use different components.

[0090] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program that, when executed, implements the tokamak long-pulse plasma configuration control method described in any of the above embodiments.

[0091] It should be understood that the implementation of all or part of the above-described tokamak long-pulse plasma configuration control method can also be accomplished by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of the above-described tokamak long-pulse plasma configuration control method. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0092] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. It should be noted that, for those skilled in the art, several equivalent obvious modifications and / or equivalent substitutions can be made without departing from the technical principles of the present invention, and these obvious modifications and / or equivalent substitutions should also be considered within the scope of protection of the present invention.

Claims

1. A method for configuration control of long-pulse plasma in a tokamak, characterized in that, include: Acquire real-time status information during the operation of the tokamak device. The real-time status information includes at least plasma configuration parameters, current safety margin of each poloidal field coil, and historical time-series status data. The real-time state information is input into a pre-trained reinforcement learning agent, which then outputs the voltage control quantity corresponding to each pole field coil. The voltage control quantity is applied to the corresponding poloidal field coils to achieve plasma closed-loop configuration control.

2. The tokamak long-pulse plasma configuration control method as described in claim 1, characterized in that, The normalized value range of the current safety margin is [-1, 1]. The sign of the margin indicates the direction in which the coil current approaches the risk of overcurrent, and the absolute value of the margin indicates the urgency of the overcurrent risk. Among them, a margin of 0 indicates that the coil current is in the center of the safe range, a margin of 1 indicates that a positive overcurrent has occurred, and a margin of -1 indicates that a negative overcurrent has occurred.

3. The tokamak long-pulse plasma configuration control method as described in claim 1, characterized in that, The training phase of the reinforcement learning agent adopts a multi-objective composite reward function, which includes a tracking reward term, an action penalty term, and a coil current overcurrent prevention penalty term. The tracking reward term is used to generate a reward based on the tracking error between the actual values ​​of plasma current and configuration parameters and the reference target values, so as to drive the plasma state to converge toward the target configuration. The action penalty term is used to impose penalty constraints on the amplitude and variation range of the voltage control action output by the intelligent agent, so as to obtain a smooth and controllable control output. The coil current overcurrent prevention penalty term is constructed based on the current safety margin of all poloidal field coils. It applies a progressively increasing penalty to coils that approach the overcurrent threshold. When multiple coils approach the threshold simultaneously, the penalties are superimposed, guiding the agent to achieve active allocation and management of the current margin of multiple coils.

4. The tokamak long-pulse plasma configuration control method as described in claim 1, characterized in that, The reinforcement learning agent adopts an Actor-Critic network architecture and completes policy iterative training based on a proximal policy optimization algorithm.

5. The tokamak long-pulse plasma configuration control method as described in claim 1, characterized in that, The voltage control quantities corresponding to each poloidal field coil output by the reinforcement learning agent include: Based on the real-time status information input, the safety margin of the coil current of each poloidal field coil and the plasma configuration control target are comprehensively weighed. When the current of any poloidal field coil is detected to be approaching the safety limit, the agent autonomously lowers the voltage control quantity corresponding to that coil, while raising the voltage control quantity corresponding to the other poloidal field coils with residual current margin, thereby avoiding the risk of coil overcurrent while maintaining the configuration control target.

6. The tokamak long-pulse plasma configuration control method as described in claim 1, characterized in that, The training process of the agent adopts a course learning strategy. During the training process, the difficulty of controlling the rate of change of plasma current is gradually increased, and the current safety margin constraint boundary of each pole field coil is gradually tightened, so as to progressively improve the agent's multi-coil collaborative compensation control capability under overcurrent boundary conditions and external strong disturbance conditions.

7. The tokamak long-pulse plasma configuration control method as described in claim 1, characterized in that, The method further includes: The voltage control quantity output by the intelligent agent is gain-scaled using a safe voltage limit gain vector so that the scaled voltage control quantity is limited to the safe voltage range allowed by the corresponding coil power supply hardware.

8. A tokamak long-pulse plasma configuration control device, characterized in that, include: The status acquisition module is used to acquire real-time status information during the operation of the tokamak device. The real-time status information includes at least plasma configuration parameters, current safety margin of each poloidal field coil, and historical time-series status data. The reinforcement learning module is used to input the real-time state information into a pre-trained reinforcement learning agent, and the reinforcement learning agent outputs the voltage control quantity corresponding to each pole field coil. The configuration control module is used to apply the voltage control quantity to the corresponding poloidal field coils to achieve plasma closed-loop configuration control.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program; The processor executes the computer program to implement the tokamak long-pulse plasma configuration control method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed, implements the tokamak long-pulse plasma configuration control method as described in any one of claims 1 to 7.

Images